Repository: chrovis/cljam
Pull request #290 (2023-11-14 07:17:57)

Use ByteBuffer wrapper instead of LSB protocol
This PR is the first step of #289.

It replaces the use of `ByteBuffer` as an LSB protocol implementation with a new thin wrapper utility for `ByteBuffer`.

The modification policy here is to use the bare `ByteBuffer` methods whenever possible. The new wrapper namespace only collects functions that do what a single method of `ByteBuffer` cannot do. Most of the implementation code comes from [the implementation of the LSB protocol](https://github.com/chrovis/cljam/blob/14e0deb16f4423686d5d68907e040fe9729d262d/src/cljam/io/util/lsb.clj#L36-L82).

codecov[bot]: ## [Codecov](https://app.codecov.io/gh/chrovis/cljam/pull/290?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=chrovis) Report
Attention: `3 lines` in your changes are missing coverage. Please review.
> Comparison is base [(`14e0deb`)](https://app.codecov.io/gh/chrovis/cljam/commit/14e0deb16f4423686d5d68907e040fe9729d262d?el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=chrovis) 88.81% compared to head [(`85647f8`)](https://app.codecov.io/gh/chrovis/cljam/pull/290?src=pr&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=chrovis) 88.85%.

| [Files](https://app.codecov.io/gh/chrovis/cljam/pull/290?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=chrovis) | Patch % | Lines |
|---|---|---|
| [src/cljam/io/util/byte\_buffer.clj](https://app.codecov.io/gh/chrovis/cljam/pull/290?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=chrovis#diff-c3JjL2NsamFtL2lvL3V0aWwvYnl0ZV9idWZmZXIuY2xq) | 95.34% | [2 Missing :warning: ](https://app.codecov.io/gh/chrovis/cljam/pull/290?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=chrovis) |
| [src/cljam/io/bcf/reader.clj](https://app.codecov.io/gh/chrovis/cljam/pull/290?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=chrovis#diff-c3JjL2NsamFtL2lvL2JjZi9yZWFkZXIuY2xq) | 95.65% | [1 Missing :warning: ](https://app.codecov.io/gh/chrovis/cljam/pull/290?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=chrovis) |

<details><summary>Additional details and impacted files</summary>


```diff
@@            Coverage Diff             @@
##           master     #290      +/-   ##
==========================================
+ Coverage   88.81%   88.85%   +0.04%     
==========================================
  Files          79       80       +1     
  Lines        6796     6839      +43     
  Branches      479      479              
==========================================
+ Hits         6036     6077      +41     
- Misses        281      283       +2     
  Partials      479      479              
```



</details>

[:umbrella: View full report in Codecov by Sentry](https://app.codecov.io/gh/chrovis/cljam/pull/290?src=pr&el=continue&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=chrovis).   
:loudspeaker: Have feedback on the report? [Share it here](https://about.codecov.io/codecov-pr-comment-feedback/?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=chrovis).

athos: Added accidentally omitted docstrings.
athos: Added some code to benchmark the performance of the BCF reader, and I've confirmed that the changes don't degrade the performance of the BAM/BCF readers:

- before

```
decode-bam-alignment-long-bench (sam_bench.clj:51)

  ["/Users/sohta/work/src/github.com/chrovis/cljam/.cavia/large.bam"]
  time: 21.659904 sec, sd: 62.121323 ms

decode-large-bcf-bench (vcf_bench.clj:14)

  ["/Users/sohta/work/src/github.com/chrovis/cljam/.cavia/large.bcf"]
  time: 24.589088 sec, sd: 515.149512 ms
```

- after

```
decode-bam-alignment-long-bench (sam_bench.clj:51)

  ["/Users/sohta/work/src/github.com/chrovis/cljam/.cavia/large.bam"]
  time: 19.811300 sec, sd: 63.591633 ms

decode-large-bcf-bench (vcf_bench.clj:14)

  ["/Users/sohta/work/src/github.com/chrovis/cljam/.cavia/large.bcf"]
  time: 24.433835 sec, sd: 878.578249 ms
```

Patch:
diff --git a/bench/cljam/io/vcf_bench.clj b/bench/cljam/io/vcf_bench.clj
new file mode 100644
index 0000000..a77f828
--- /dev/null
+++ b/bench/cljam/io/vcf_bench.clj
@@ -0,0 +1,19 @@
+(ns cljam.io.vcf-bench
+  (:require [cljam.io.vcf :as vcf]
+            [cljam.test-common :as tcommon]
+            [libra.bench :refer [are defbench]]
+            [libra.criterium :as c]))
+
+(defbench decode-small-bcf-bench
+  (are [f]
+       (c/quick-bench
+        (with-open [r (vcf/reader f)]
+          (run! (constantly nil) (vcf/read-variants r))))
+    tcommon/test-bcf-complex-file))
+
+(defbench decode-large-bcf-bench
+  (are [f]
+       (c/quick-bench
+        (with-open [r (vcf/reader f)]
+          (run! (constantly nil) (vcf/read-variants-randomly r {:chr "chr1" :end 30000000} {}))))
+    tcommon/test-large-bcf-file))
diff --git a/src/cljam/io/bam/decoder.clj b/src/cljam/io/bam/decoder.clj
index 9589890..6f1c86d 100644
--- a/src/cljam/io/bam/decoder.clj
+++ b/src/cljam/io/bam/decoder.clj
@@ -8,7 +8,7 @@
             [cljam.io.sam.util.refs :as refs]
             [cljam.io.sam.util.cigar :as cigar]
             [cljam.io.bam.common :as common]
-            [cljam.io.util.lsb :as lsb])
+            [cljam.io.util.byte-buffer :as bb])
   (:import [java.util Arrays]
            [java.nio Buffer ByteBuffer ByteOrder CharBuffer]
            [cljam.io.protocols SAMAlignment SAMRegionBlock SAMCoordinateBlock SAMQuerynameBlock]))
@@ -28,7 +28,7 @@
   "Parses a tag according to `tag-type`."
   [tag-type ^ByteBuffer bb]
   `(case (long ~tag-type)
-     ~(long \Z) (lsb/read-null-terminated-string ~bb)
+     ~(long \Z) (bb/read-null-terminated-string ~bb)
      ~(long \A) (char (.get ~bb))
      ~(long \I) (bit-and (.getInt ~bb) 0xffffffff)
      ~(long \i) (.getInt ~bb)
@@ -37,7 +37,7 @@
      ~(long \c) (int (.get ~bb))
      ~(long \C) (bit-and (int (.get ~bb)) 0xff)
      ~(long \f) (.getFloat ~bb)
-     ~(long \H) (proton/hex->bytes (lsb/read-null-terminated-string ~bb))
+     ~(long \H) (proton/hex->bytes (bb/read-null-terminated-string ~bb))
      (throw (Exception. "Unrecognized tag type"))))
 
 (defn- parse-tag-array [^ByteBuffer bb]
@@ -129,28 +129,28 @@
   When called with start and end, this function may return nil if any base of the block
   is not included in the range."
   ([refs block]
-   (let [buffer      (ByteBuffer/wrap (:data block))
-         ref-id      ^int (lsb/read-int buffer)
+   (let [buffer      (bb/make-lsb-byte-buffer (:data block))
+         ref-id      (.getInt buffer)
          rname       (or (refs/ref-name refs ref-id) "*")
-         pos         (inc (int (lsb/read-int buffer)))
-         l-read-name (short (lsb/read-ubyte buffer))
-         mapq        (short (lsb/read-ubyte buffer))
-         _           (lsb/skip buffer 2) ; bin
-         n-cigar-op  (int (lsb/read-ushort buffer))
-         flag        (int (lsb/read-ushort buffer))
-         l-seq       (int (lsb/read-int buffer))
-         next-ref-id (int (lsb/read-int buffer))
+         pos         (inc (.getInt buffer))
+         l-read-name (short (bb/read-ubyte buffer))
+         mapq        (short (bb/read-ubyte buffer))
+         _           (bb/skip buffer 2) ; bin
+         n-cigar-op  (int (bb/read-ushort buffer))
+         flag        (int (bb/read-ushort buffer))
+         l-seq       (.getInt buffer)
+         next-ref-id (.getInt buffer)
          rnext       (decode-next-ref-id refs ref-id next-ref-id)
-         pnext       (inc (int (lsb/read-int buffer)))
-         tlen        (int (lsb/read-int buffer))
-         qname       (lsb/read-string buffer (dec l-read-name))
-         _           (lsb/skip buffer 1)
-         cigar-bytes (lsb/read-bytes buffer (* n-cigar-op 4))
+         pnext       (inc (.getInt buffer))
+         tlen        (.getInt buffer)
+         qname       (bb/read-string buffer (dec l-read-name))
+         _           (bb/skip buffer 1)
+         cigar-bytes (bb/read-bytes buffer (* n-cigar-op 4))
          [cigar len] (cigar/decode-cigar-and-ref-length cigar-bytes)
          ref-end     (if (zero? (long len)) pos (dec (+ pos (long len))))
-         seq'        (decode-seq (lsb/read-bytes buffer (quot (inc l-seq) 2)) l-seq)
-         qual        (decode-qual (lsb/read-bytes buffer l-seq))
-         rest'        (lsb/read-bytes buffer (options-size (alength ^bytes (:data block)) l-read-name n-cigar-op l-seq))
+         seq'        (decode-seq (bb/read-bytes buffer (quot (inc l-seq) 2)) l-seq)
+         qual        (decode-qual (bb/read-bytes buffer l-seq))
+         rest'        (bb/read-bytes buffer (options-size (alength ^bytes (:data block)) l-read-name n-cigar-op l-seq))
          options     (decode-options rest')
          [cigar* options*]
          (if-let [cg (and (cigar/placeholder? cigar-bytes)
@@ -160,31 +160,31 @@
      (SAMAlignment. qname (int flag) rname (int pos) ref-end (int mapq)
                     cigar* rnext (int pnext) (int tlen) seq' qual options*)))
   ([refs block ^long start ^long end]
-   (let [buffer          (ByteBuffer/wrap (:data block))
-         ref-id          (int (lsb/read-int buffer))
-         pos             (inc (int (lsb/read-int buffer)))]
+   (let [buffer          (bb/make-lsb-byte-buffer (:data block))
+         ref-id          (.getInt buffer)
+         pos             (inc (.getInt buffer))]
      (when (<= pos end)
-       (let [l-read-name (short (lsb/read-ubyte buffer))
-             mapq        (short (lsb/read-ubyte buffer))
-             _           (lsb/skip buffer 2) ; bin
-             n-cigar-op  (int (lsb/read-ushort buffer))
-             flag        (int (lsb/read-ushort buffer))
-             l-seq       (int (lsb/read-int buffer))
-             next-ref-id (int (lsb/read-int buffer))
+       (let [l-read-name (short (bb/read-ubyte buffer))
+             mapq        (short (bb/read-ubyte buffer))
+             _           (bb/skip buffer 2) ; bin
+             n-cigar-op  (int (bb/read-ushort buffer))
+             flag        (int (bb/read-ushort buffer))
+             l-seq       (.getInt buffer)
+             next-ref-id (.getInt buffer)
              rnext       (decode-next-ref-id refs ref-id next-ref-id)
-             pnext       (inc (int (lsb/read-int buffer)))
-             tlen        (int (lsb/read-int buffer))
-             qname       (lsb/read-string buffer (dec l-read-name))
-             _           (lsb/skip buffer 1)
-             cigar-bytes (lsb/read-bytes buffer (* n-cigar-op 4))
+             pnext       (inc (.getInt buffer))
+             tlen        (.getInt buffer)
+             qname       (bb/read-string buffer (dec l-read-name))
+             _           (bb/skip buffer 1)
+             cigar-bytes (bb/read-bytes buffer (* n-cigar-op 4))
              [cigar len] (cigar/decode-cigar-and-ref-length cigar-bytes)
              ref-end     (int (if (zero? (long len))
                                 pos
                                 (dec (+ pos (long len)))))]
          (when (<= start ref-end)
-           (let [seq'    (decode-seq (lsb/read-bytes buffer (quot (inc l-seq) 2)) l-seq)
-                 qual    (decode-qual (lsb/read-bytes buffer l-seq))
-                 rest'    (lsb/read-bytes buffer (options-size (alength ^bytes (:data block)) l-read-name n-cigar-op l-seq))
+           (let [seq'    (decode-seq (bb/read-bytes buffer (quot (inc l-seq) 2)) l-seq)
+                 qual    (decode-qual (bb/read-bytes buffer l-seq))
+                 rest'    (bb/read-bytes buffer (options-size (alength ^bytes (:data block)) l-read-name n-cigar-op l-seq))
                  rname   (or (refs/ref-name refs ref-id) "*")
                  options (decode-options rest')
                  [cigar* options*]
@@ -198,27 +198,27 @@
 (defn decode-region-block
   "Decodes BAM block and returns a SAMRegionBlock instance containing covering range of the alignment."
   ([^BAMRawBlock block]
-   (let [buffer          (ByteBuffer/wrap (.data block))
-         ref-id          (int (lsb/read-int buffer))
-         pos             (inc (int (lsb/read-int buffer)))
-         l-read-name     (short (lsb/read-ubyte buffer))
-         _               (lsb/skip buffer 3) ;; MAPQ, bin
-         n-cigar-op      (int (lsb/read-ushort buffer))
-         _               (lsb/skip buffer (+ 18 l-read-name)) ;; flag, l_seq, rnext, pnext, tlen, qname
-         cigar-bytes     (lsb/read-bytes buffer (* n-cigar-op 4))
+   (let [buffer          (bb/make-lsb-byte-buffer (.data block))
+         ref-id          (.getInt buffer)
+         pos             (inc (.getInt buffer))
+         l-read-name     (short (bb/read-ubyte buffer))
+         _               (bb/skip buffer 3) ;; MAPQ, bin
+         n-cigar-op      (int (bb/read-ushort buffer))
+         _               (bb/skip buffer (+ 18 l-read-name)) ;; flag, l_seq, rnext, pnext, tlen, qname
+         cigar-bytes     (bb/read-bytes buffer (* n-cigar-op 4))
          ref-length      (cigar/count-ref-bytes cigar-bytes)
          ref-end         (int (if (zero? ref-length) pos (dec (+ pos ref-length))))]
      (SAMRegionBlock. (.data block) ref-id pos ref-end)))
   ([^BAMRawBlock block ^long start ^long end]
-   (let [buffer          (ByteBuffer/wrap (.data block))
-         ref-id          (int (lsb/read-int buffer))
-         pos             (inc (int (lsb/read-int buffer)))]
+   (let [buffer          (bb/make-lsb-byte-buffer (.data block))
+         ref-id          (.getInt buffer)
+         pos             (inc (.getInt buffer))]
      (when (<= pos end)
-       (let [l-read-name (short (lsb/read-ubyte buffer))
-             _           (lsb/skip buffer 3) ;; MAPQ, bin
-             n-cigar-op  (int (lsb/read-ushort buffer))
-             _           (lsb/skip buffer (+ 18 l-read-name)) ;; flag, l_seq, rnext, pnext, tlen, qname
-             cigar-bytes (lsb/read-bytes buffer (* n-cigar-op 4))
+       (let [l-read-name (short (bb/read-ubyte buffer))
+             _           (bb/skip buffer 3) ;; MAPQ, bin
+             n-cigar-op  (int (bb/read-ushort buffer))
+             _           (bb/skip buffer (+ 18 l-read-name)) ;; flag, l_seq, rnext, pnext, tlen, qname
+             cigar-bytes (bb/read-bytes buffer (* n-cigar-op 4))
              ref-length  (cigar/count-ref-bytes cigar-bytes)
              ref-end     (if (zero? ref-length) pos (dec (+ pos ref-length)))]
          (when (<= start ref-end)
@@ -227,23 +227,23 @@
 (defn decode-coordinate-block
   "Decodes BAM block and returns a SAMCoordinateBlock instance containing ref-id, pos and flag."
   ([^BAMRawBlock block]
-   (let [buffer     (ByteBuffer/wrap (.data block))
-         ref-id     (int (lsb/read-int buffer))
-         pos        (inc (int (lsb/read-int buffer)))
-         _          (lsb/skip buffer 6) ;; l_read_name, MAPQ, bin, n_cigar_op
-         flag       (int (lsb/read-ushort buffer))] ;; l_seq, rnext, pnext, tlen, qname
+   (let [buffer     (bb/make-lsb-byte-buffer (.data block))
+         ref-id     (.getInt buffer)
+         pos        (inc (.getInt buffer))
+         _          (bb/skip buffer 6) ;; l_read_name, MAPQ, bin, n_cigar_op
+         flag       (int (bb/read-ushort buffer))] ;; l_seq, rnext, pnext, tlen, qname
      (SAMCoordinateBlock. (.data block) (int ref-id) (int pos) (int flag))))
   ([^BAMRawBlock block ^long start ^long end]
-   (let [buffer          (ByteBuffer/wrap (.data block))
-         ref-id          (int (lsb/read-int buffer))
-         pos             (inc (int (lsb/read-int buffer)))]
+   (let [buffer          (bb/make-lsb-byte-buffer (.data block))
+         ref-id          (.getInt buffer)
+         pos             (inc (.getInt buffer))]
      (when (<= pos end)
-       (let [l-read-name (short (lsb/read-ubyte buffer))
-             _           (lsb/skip buffer 3) ;; MAPQ, bin
-             n-cigar-op  (int (lsb/read-ushort buffer))
-             flag        (int (lsb/read-ushort buffer))
-             _           (lsb/skip buffer (+ 16 l-read-name)) ;; l_seq, rnext, pnext, tlen, qname
-             cigar-bytes (lsb/read-bytes buffer (* n-cigar-op 4))
+       (let [l-read-name (short (bb/read-ubyte buffer))
+             _           (bb/skip buffer 3) ;; MAPQ, bin
+             n-cigar-op  (int (bb/read-ushort buffer))
+             flag        (int (bb/read-ushort buffer))
+             _           (bb/skip buffer (+ 16 l-read-name)) ;; l_seq, rnext, pnext, tlen, qname
+             cigar-bytes (bb/read-bytes buffer (* n-cigar-op 4))
              ref-length  (cigar/count-ref-bytes cigar-bytes)
              ref-end     (if (zero? ref-length) pos (dec (+ pos ref-length)))]
          (when (<= start ref-end)
@@ -252,26 +252,26 @@
 (defn decode-queryname-block
   "Decodes BAM block and returns a SAMQuerynameBlock instance containing qname and flag."
   ([^BAMRawBlock block]
-   (let [buffer      (ByteBuffer/wrap (.data block))
-         _           (lsb/skip buffer 8) ;; ref-id, pos
-         l-read-name (short (lsb/read-ubyte buffer))
-         _           (lsb/skip buffer 5) ;; MAPQ, bin, n_cigar_op
-         flag        (int (lsb/read-ushort buffer))
-         _           (lsb/skip buffer 16) ;; l_seq, rnext, pnext, tlen
-         qname       (lsb/read-string buffer l-read-name)]
+   (let [buffer      (bb/make-lsb-byte-buffer (.data block))
+         _           (bb/skip buffer 8) ;; ref-id, pos
+         l-read-name (short (bb/read-ubyte buffer))
+         _           (bb/skip buffer 5) ;; MAPQ, bin, n_cigar_op
+         flag        (int (bb/read-ushort buffer))
+         _           (bb/skip buffer 16) ;; l_seq, rnext, pnext, tlen
+         qname       (bb/read-string buffer l-read-name)]
      (SAMQuerynameBlock. (.data block) qname (int flag))))
   ([^BAMRawBlock block ^long start ^long end]
-   (let [buffer          (ByteBuffer/wrap (.data block))
-         _               (lsb/skip buffer 4) ;; ref-id
-         pos             (inc (int (lsb/read-int buffer)))]
+   (let [buffer          (bb/make-lsb-byte-buffer (.data block))
+         _               (bb/skip buffer 4) ;; ref-id
+         pos             (inc (.getInt buffer))]
      (when (<= pos end)
-       (let [l-read-name (short (lsb/read-ubyte buffer))
-             _           (lsb/skip buffer 3) ;; MAPQ, bin
-             n-cigar-op  (int (lsb/read-ushort buffer))
-             flag        (int (lsb/read-ushort buffer))
-             _           (lsb/skip buffer 16) ;; l_seq, rnext, pnext, tlen
-             qname       (lsb/read-string buffer l-read-name)
-             cigar-bytes (lsb/read-bytes buffer (* n-cigar-op 4))
+       (let [l-read-name (short (bb/read-ubyte buffer))
+             _           (bb/skip buffer 3) ;; MAPQ, bin
+             n-cigar-op  (int (bb/read-ushort buffer))
+             flag        (int (bb/read-ushort buffer))
+             _           (bb/skip buffer 16) ;; l_seq, rnext, pnext, tlen
+             qname       (bb/read-string buffer l-read-name)
+             cigar-bytes (bb/read-bytes buffer (* n-cigar-op 4))
              ref-length  (cigar/count-ref-bytes cigar-bytes)
              ref-end     (if (zero? ref-length) pos (dec (+ pos ref-length)))]
          (when (<= start ref-end)
@@ -281,29 +281,29 @@
 (defn decode-pointer-block
   "Decodes BAM block and returns a BAMPointerBlock instance containing region, flag and block pointers."
   ([^BAMRawBlock block]
-   (let [buffer      (ByteBuffer/wrap (.data block))
-         ref-id      (int (lsb/read-int buffer))
-         pos         (inc (int (lsb/read-int buffer)))
-         l-read-name (short (lsb/read-ubyte buffer))
-         _           (lsb/skip buffer 3) ;; MAPQ, bin
-         n-cigar-op  (int (lsb/read-ushort buffer))
-         flag        (int (lsb/read-ushort buffer))
-         _           (lsb/skip buffer (+ 16 l-read-name)) ;; l_seq, rnext, pnext, tlen, qname
-         cigar-bytes (lsb/read-bytes buffer (* n-cigar-op 4))
+   (let [buffer      (bb/make-lsb-byte-buffer (.data block))
+         ref-id      (.getInt buffer)
+         pos         (inc (.getInt buffer))
+         l-read-name (short (bb/read-ubyte buffer))
+         _           (bb/skip buffer 3) ;; MAPQ, bin
+         n-cigar-op  (int (bb/read-ushort buffer))
+         flag        (int (bb/read-ushort buffer))
+         _           (bb/skip buffer (+ 16 l-read-name)) ;; l_seq, rnext, pnext, tlen, qname
+         cigar-bytes (bb/read-bytes buffer (* n-cigar-op 4))
          ref-length  (cigar/count-ref-bytes cigar-bytes)
          ref-end     (if (zero? ref-length) pos (dec (+ pos ref-length)))]
      (BAMPointerBlock. (.data block) ref-id pos ref-end (int flag) (.pointer-beg block) (.pointer-end block))))
   ([^BAMRawBlock block ^long start ^long end]
-   (let [buffer          (ByteBuffer/wrap (.data block))
-         ref-id          (int (lsb/read-int buffer))
-         pos             (inc (int (lsb/read-int buffer)))]
+   (let [buffer          (bb/make-lsb-byte-buffer (.data block))
+         ref-id          (.getInt buffer)
+         pos             (inc (.getInt buffer))]
      (when (<= pos end)
-       (let [l-read-name (short (lsb/read-ubyte buffer))
-             _           (lsb/skip buffer 3) ;; MAPQ, bin
-             n-cigar-op  (int (lsb/read-ushort buffer))
-             flag        (int (lsb/read-ushort buffer))
-             _           (lsb/skip buffer (+ 16 l-read-name)) ;; l_seq, rnext, pnext, tlen, qname
-             cigar-bytes (lsb/read-bytes buffer (* n-cigar-op 4))
+       (let [l-read-name (short (bb/read-ubyte buffer))
+             _           (bb/skip buffer 3) ;; MAPQ, bin
+             n-cigar-op  (int (bb/read-ushort buffer))
+             flag        (int (bb/read-ushort buffer))
+             _           (bb/skip buffer (+ 16 l-read-name)) ;; l_seq, rnext, pnext, tlen, qname
+             cigar-bytes (bb/read-bytes buffer (* n-cigar-op 4))
              ref-length  (cigar/count-ref-bytes cigar-bytes)
              ref-end     (if (zero? ref-length) pos (dec (+ pos ref-length)))]
          (when (<= start ref-end)
diff --git a/src/cljam/io/bcf/reader.clj b/src/cljam/io/bcf/reader.clj
index 9c62580..9449543 100644
--- a/src/cljam/io/bcf/reader.clj
+++ b/src/cljam/io/bcf/reader.clj
@@ -3,6 +3,7 @@
             [clojure.tools.logging :as logging]
             [cljam.io.protocols :as protocols]
             [cljam.io.util.bgzf :as bgzf]
+            [cljam.io.util.byte-buffer :as bb]
             [cljam.io.util.lsb :as lsb]
             [cljam.io.vcf.reader :as vcf-reader]
             [cljam.io.vcf.util :as vcf-util]
@@ -110,18 +111,18 @@
 (defn- read-typed-atomic-value
   "Reads an atomic value, which is typed as either
   integer(8,16,32 bit) or float or character."
-  [r ^long type-id]
+  [^ByteBuffer bb ^long type-id]
   (case type-id
-    1 (let [i (byte (lsb/read-byte r))]
+    1 (let [i (.get bb)]
         (case (bit-and 0xFF i) 0x80 nil 0x81 :eov i))
-    2 (let [i (short (lsb/read-short r))]
+    2 (let [i (.getShort bb)]
         (case (bit-and 0xFFFF i) 0x8000 nil 0x8001 :eov i))
-    3 (let [i (int (lsb/read-int r))]
+    3 (let [i (.getInt bb)]
         (case (bit-and 0xFFFFFFFF i) 0x80000000 nil 0x80000001 :eov i))
-    5 (let [i (int (lsb/read-int r))]
+    5 (let [i (.getInt bb)]
         (case (bit-and 0xFFFFFFFF i) 0x7F800001 nil 0x7F800002
               :eov (Float/intBitsToFloat i)))
-    7 (lsb/read-byte r)))
+    7 (.get bb)))
 
 (defn- bytes->strs
   [ba]
@@ -132,20 +133,20 @@
        (cstr/split (String. (byte-array ba)) #",")))
 
 (defn- read-typed-value
-  "Reads typed value from BCF file. n-sample is a number of values repeated
+  "Reads typed value from byte buffer. n-sample is a number of values repeated
   after type specifier byte."
-  ([rdr]
-   (first (read-typed-value rdr 1)))
-  ([rdr ^long n-sample]
-   (let [type-byte (int (lsb/read-byte rdr))
+  ([bb]
+   (first (read-typed-value bb 1)))
+  ([^ByteBuffer bb ^long n-sample]
+   (let [type-byte (int (.get bb))
          len (unsigned-bit-shift-right (bit-and 0xF0 type-byte) 4)
-         total-len (if (= len 15) (long (first (read-typed-value rdr))) len)
+         total-len (if (= len 15) (long (first (read-typed-value bb))) len)
          type-id (bit-and 0x0F type-byte)]
      (case type-id
        0 (repeat n-sample nil)
        7 (doall (repeatedly n-sample
-                            #(bytes->strs (lsb/read-bytes rdr total-len))))
-       (->> #(read-typed-atomic-value rdr type-id)
+                            #(bytes->strs (bb/read-bytes bb total-len))))
+       (->> #(read-typed-atomic-value bb type-id)
             (repeatedly (* n-sample total-len))
             (partition total-len)
             (map (fn [xs] (take-while #(not= % :eov) xs)))
@@ -153,19 +154,19 @@
 
 (defn- read-typed-kv
   "Reads a key-value pair."
-  ([rdr]
-   (let [[k [v]] (read-typed-kv rdr 1)]
+  ([bb]
+   (let [[k [v]] (read-typed-kv bb 1)]
      [k v]))
-  ([rdr n-sample]
-   [(first (read-typed-value rdr)) (read-typed-value rdr n-sample)]))
+  ([bb n-sample]
+   [(first (read-typed-value bb)) (read-typed-value bb n-sample)]))
 
 (defn- read-data-line-buffer
   "Reads a single record of variant and store to the ByteBuffer objects."
   [rdr]
   (let [l-shared (lsb/read-uint rdr)
         l-indv (lsb/read-uint rdr)
-        shared-bb (ByteBuffer/allocate l-shared)
-        indv-bb (ByteBuffer/allocate l-indv)]
+        shared-bb (bb/allocate-lsb-byte-buffer l-shared)
+        indv-bb (bb/allocate-lsb-byte-buffer l-indv)]
     (lsb/read-bytes rdr (.array shared-bb) 0 l-shared)
     (lsb/read-bytes rdr (.array indv-bb) 0 l-indv)
     {:l-shared l-shared
@@ -176,23 +177,23 @@
 (defn- parse-data-line-shallow
   "Parses only chromosome, position and ref-length. Can be used for position-based querying."
   [contigs {:keys [^ByteBuffer shared] :as m}]
-  (let [chrom-id (lsb/read-int shared)
-        pos (inc (int (lsb/read-int shared)))
-        rlen (lsb/read-int shared)]
+  (let [chrom-id (.getInt shared)
+        pos (inc (.getInt shared))
+        rlen (.getInt shared)]
     (.position ^Buffer shared 0)
     (assoc m :chr (:id (contigs chrom-id)) :pos pos :rlen rlen)))
 
 (defn- parse-data-line-deep
   "Parses full data of a variant. Returns a map containing indices for meta-info."
   [{:keys [^ByteBuffer shared ^ByteBuffer individual]}]
-  (let [chrom-id (lsb/read-int shared)
-        pos (inc (int (lsb/read-int shared)))
-        rlen (lsb/read-int shared)
-        qual (lsb/read-int shared)
-        n-allele-info (int (lsb/read-int shared))
+  (let [chrom-id (.getInt shared)
+        pos (inc (.getInt shared))
+        rlen (.getInt shared)
+        qual (.getInt shared)
+        n-allele-info (.getInt shared)
         n-allele (unsigned-bit-shift-right n-allele-info 16)
         n-info (bit-and n-allele-info 0xFFFF)
-        n-fmt-sample (long (lsb/read-uint shared))
+        n-fmt-sample (long (bb/read-uint shared))
         n-fmt (bit-and 0xFF (unsigned-bit-shift-right n-fmt-sample 24))
         n-sample (bit-and n-fmt-sample 0xFFFFFF)
         id (let [i (read-typed-value shared)] (if (sequential? i) (first i) i))
@@ -293,7 +294,7 @@
      :vcf     VCF-style map. FORMAT, FILTER, INFO and samples columns are strings.
      :bcf     BCF-style map. CHROM, FILTER, INFO and :genotype contains indices to meta-info.
      :shallow Only CHROM, POS and ref-length are parsed.
-     :raw     Raw map of ByteBufers."
+     :raw     Raw map of ByteBuffers."
   ([rdr]
    (read-variants rdr {}))
   ([^BCFReader rdr {:keys [depth] :or {depth :deep}}]
diff --git a/src/cljam/io/util/byte_buffer.clj b/src/cljam/io/util/byte_buffer.clj
new file mode 100644
index 0000000..6fc00d6
--- /dev/null
+++ b/src/cljam/io/util/byte_buffer.clj
@@ -0,0 +1,94 @@
+(ns cljam.io.util.byte-buffer
+  (:refer-clojure :exclude [read-string])
+  (:require [cljam.util :refer [string->bytes]])
+  (:import [java.nio Buffer ByteBuffer ByteOrder]))
+
+(defn make-lsb-byte-buffer
+  "Creates a new little-endian byte buffer wrapping given data."
+  ^ByteBuffer [^bytes data]
+  (.order (ByteBuffer/wrap data) ByteOrder/LITTLE_ENDIAN))
+
+(defn make-msb-byte-buffer
+  "Creates a new big-endian byte buffer wrapping given data."
+  ^ByteBuffer [^bytes data]
+  (.order (ByteBuffer/wrap data) ByteOrder/BIG_ENDIAN))
+
+(defn allocate-lsb-byte-buffer
+  "Creates a new little-endian byte buffer with given capacity."
+  (^ByteBuffer []
+   (.order (ByteBuffer/allocate 8) ByteOrder/LITTLE_ENDIAN))
+  (^ByteBuffer [size]
+   (.order (ByteBuffer/allocate (int size)) ByteOrder/LITTLE_ENDIAN)))
+
+(defn allocate-msb-byte-buffer
+  "Creates a new big-endian byte buffer with given capacity."
+  (^ByteBuffer []
+   (.order (ByteBuffer/allocate 8) ByteOrder/BIG_ENDIAN))
+  (^ByteBuffer [size]
+   (.order (ByteBuffer/allocate (int size)) ByteOrder/BIG_ENDIAN)))
+
+(defn skip
+  "Skips over 'length' bytes of data, discarding the skipped bytes."
+  [^ByteBuffer bb ^long length]
+  (.position ^Buffer bb (+ (.position bb) length)))
+
+(defn read-ubyte
+  "Reads 1 byte. Returns an unsigned byte value as long."
+  [^ByteBuffer bb]
+  (bit-and (.get bb) 0xFF))
+
+(defn read-ushort
+  "Reads 2 bytes. Returns an unsigned short value as long."
+  [^ByteBuffer bb]
+  (bit-and (.getShort bb) 0xFFFF))
+
+(defn read-uint
+  "Reads 4 bytes. Returns an unsigned int value as long."
+  [^ByteBuffer bb]
+  (bit-and (.getInt bb) 0xFFFFFFFF))
+
+(defn read-bytes
+  "Reads 'length' bytes to buffer starting from offset bytes. Returns a new byte-array if called without buffer."
+  ([^ByteBuffer bb ^long length]
+   (let [ba (byte-array length)]
+     (.get bb ba)
+     ba))
+  ([^ByteBuffer bb buffer ^long offset ^long length]
+   (.get bb buffer (int offset) (int length))
+   buffer))
+
+(defn read-string
+  "Reads 'length' bytes. Returns a String."
+  [^ByteBuffer bb ^long length]
+  (let [ba (byte-array length)]
+    (.get bb ba)
+    (String. ba)))
+
+(defn read-null-terminated-string
+  "Reads until next null character. Returns a String without the null."
+  [^ByteBuffer bb]
+  (let [start (.position bb)
+        end (do (while (not (zero? (.get bb))))
+                (.position bb))
+        offset (.arrayOffset bb)]
+    (String. (.array bb) (+ offset start) (dec (- end start)))))
+
+(defn write-ubyte
+  "Writes 1 byte."
+  [^ByteBuffer bb b]
+  (.put bb (unchecked-byte b)))
+
+(defn write-ushort
+  "Writes a 2-byte unsigned short value."
+  [^ByteBuffer bb n]
+  (.putShort bb (unchecked-short n)))
+
+(defn write-uint
+  "Writes a 4-byte unsigned integer value."
+  [^ByteBuffer bb n]
+  (.putInt bb (unchecked-int n)))
+
+(defn write-string
+  "Writes a string as a sequence of ascii characters."
+  [^ByteBuffer bb s]
+  (.put bb (string->bytes s)))


Test patch:
diff --git a/test/cljam/io/util/byte_buffer_test.clj b/test/cljam/io/util/byte_buffer_test.clj
new file mode 100644
index 0000000..340f25c
--- /dev/null
+++ b/test/cljam/io/util/byte_buffer_test.clj
@@ -0,0 +1,112 @@
+(ns cljam.io.util.byte-buffer-test
+  (:require [cljam.io.util.byte-buffer :as bb]
+            [clojure.test :refer [deftest is testing]]))
+
+(deftest make-lsb-byte-buffer-test
+  (let [bb (bb/make-lsb-byte-buffer (byte-array [0x01 0x23 0x45 0x67]))]
+    (is (= 0x67452301 (.getInt bb)))))
+
+(deftest make-msb-byte-buffer-test
+  (let [bb (bb/make-msb-byte-buffer (byte-array [0x01 0x23 0x45 0x67]))]
+    (is (= 0x01234567 (.getInt bb)))))
+
+(deftest allocate-lsb-byte-buffer-test
+  (let [bb (doto (bb/allocate-lsb-byte-buffer 8)
+             (.putLong 0x789ABCDEF0123456)
+             (.flip))]
+    (is (= (int 0xF0123456) (.getInt bb)))
+    (is (= (int 0x789ABCDE) (.getInt bb)))))
+
+(deftest allocate-msb-byte-buffer-test
+  (let [bb (doto (bb/allocate-msb-byte-buffer 8)
+             (.putLong 0x789ABCDEF0123456)
+             (.flip))]
+    (is (= (int 0x789ABCDE) (.getInt bb)))
+    (is (= (int 0xF0123456) (.getInt bb)))))
+
+(deftest read-ops-test
+  (let [bb (doto (bb/allocate-lsb-byte-buffer 8)
+             (.mark)
+             (.putLong 0x789ABCDEF0123456))]
+    (testing "read-ubyte"
+      (.reset bb)
+      (is (= 0x56 (bb/read-ubyte bb)))
+      (is (= 0x34 (bb/read-ubyte bb)))
+      (is (= 0x12 (bb/read-ubyte bb)))
+      (is (= 0xF0 (bb/read-ubyte bb)))
+      (is (= 0xDE (bb/read-ubyte bb)))
+      (is (= 0xBC (bb/read-ubyte bb)))
+      (is (= 0x9A (bb/read-ubyte bb)))
+      (is (= 0x78 (bb/read-ubyte bb))))
+
+    (testing "read-ushort"
+      (.reset bb)
+      (is (= 0x3456 (bb/read-ushort bb)))
+      (is (= 0xF012 (bb/read-ushort bb)))
+      (is (= 0xBCDE (bb/read-ushort bb)))
+      (is (= 0x789A (bb/read-ushort bb))))
+
+    (testing "read-unit"
+      (.reset bb)
+      (is (= 0xF0123456 (bb/read-uint bb)))
+      (is (= 0x789ABCDE (bb/read-uint bb))))
+
+    (testing "read-bytes"
+      (.reset bb)
+      (is (= [0x56 0x34 0x12 0xF0 0xDE 0xBC 0x9A 0x78]
+             (map #(bit-and % 0xFF) (bb/read-bytes bb 8))))
+
+      (.reset bb)
+      (bb/skip bb 1)
+      (is (= [0 0 0x34 0x12 0xF0 0]
+             (map #(bit-and % 0xFF) (bb/read-bytes bb (byte-array 6) 2 3)))))
+
+    (testing "read-string"
+      (.reset bb)
+      (doseq [c "ABCDEFGH"]
+        (.put bb (byte (int c))))
+      (.reset bb)
+      (is (= "ABCDEFGH" (bb/read-string bb 8))))
+
+    (testing "read-null-terminated-string"
+      (.reset bb)
+      (doseq [c [\I \J \K \L 0 \M \N \O]]
+        (.put bb (byte (int c))))
+      (.reset bb)
+      (is (= "IJKL" (bb/read-null-terminated-string bb))))))
+
+(deftest write-ops-test
+  (let [bb (bb/allocate-msb-byte-buffer 8)]
+    (testing "write-ubyte"
+      (bb/write-ubyte bb 0x78)
+      (bb/write-ubyte bb 0x9A)
+      (bb/write-ubyte bb 0xBC)
+      (bb/write-ubyte bb 0xDE)
+      (bb/write-ubyte bb 0xF0)
+      (bb/write-ubyte bb 0x12)
+      (bb/write-ubyte bb 0x34)
+      (bb/write-ubyte bb 0x56)
+      (.flip bb)
+      (is (= 0x789ABCDEF0123456 (.getLong bb))))
+
+    (testing "write-ushort"
+      (.flip bb)
+      (bb/write-ushort bb 0x789A)
+      (bb/write-ushort bb 0xBCDE)
+      (bb/write-ushort bb 0xF012)
+      (bb/write-ushort bb 0x3456)
+      (.flip bb)
+      (is (= 0x789ABCDEF0123456 (.getLong bb))))
+
+    (testing "write-uint"
+      (.flip bb)
+      (bb/write-uint bb 0x789ABCDE)
+      (bb/write-uint bb 0xF0123456)
+      (.flip bb)
+      (is (= 0x789ABCDEF0123456 (.getLong bb))))
+
+    (testing "write-string"
+      (.flip bb)
+      (bb/write-string bb "ABCDEFGH")
+      (.flip bb)
+      (is (= "ABCDEFGH" (bb/read-string bb 8))))))