Repository: biopython/biopython

Issue:
Unable to bgzip fastq file with bgzf.py
### Setup

I am reporting a problem with Biopython version, Python version, and operating
system as follows:

```python
import sys; print(sys.version)
3.5.5 | packaged by conda-forge | (default, Feb 13 2018, 05:02:37) 
[GCC 4.8.2 20140120 (Red Hat 4.8.2-15)]
import platform; print(platform.python_implementation()); print(platform.platform())
CPython
Linux-4.9.0-6-amd64-x86_64-with-debian-9.4
import Bio; print(Bio.__version__)
1.70
```

### Expected behaviour

I expect that `cat test.fastq | python bgzf.py > test.fastq.bgz` terminates with status code 0 an creates the file `test.fastq.bgz`.

### Actual behaviour

```
cat test.fastq | python bgzf.py > test.fastq.bgz
Producing BGZF output from stdin...
Traceback (most recent call last):
  File "/home/jens/miniconda3/envs/sc-preprocess/lib/python3.5/site-packages/Bio/bgzf.py", line 878, in <module>
    w.close()
  File "/home/jens/miniconda3/envs/sc-preprocess/lib/python3.5/site-packages/Bio/bgzf.py", line 822, in close
    self.flush()
  File "/home/jens/miniconda3/envs/sc-preprocess/lib/python3.5/site-packages/Bio/bgzf.py", line 809, in flush
    self._write_block(self._buffer)
  File "/home/jens/miniconda3/envs/sc-preprocess/lib/python3.5/site-packages/Bio/bgzf.py", line 785, in _write_block
    self._handle.write(data)
TypeError: write() argument must be str, not bytes
```

The file `test.fastq.bgz` exists, but has size 0.

### Steps to reproduce

Create a file `test.fastq` with the following content:
```
@NS500475:188:HW5KCBGXY:1:11101:9009:7600:CAATACAAAGG 2:N:0:1
TTCCAACACAGCCCAATATCAAAAAATAAAAACAAAAAAAAACCAAAAAAAAGAATAAGTATTAAAATCAGGTTATTAA
+
///A////<<E/A///////EAEEEE/EEE/AE6666/EE///EE//AEA/EAEE//E////EA///EA/E////</E/
@NS500475:188:HW5KCBGXY:1:11101:26461:7600:CGAAACGAATT 2:N:0:1
GGGTTTGAGTGAGGTGATGGGGAGGGTGGAGTTGTTGGATGATAGCAGCATTCTATACTTAATATAAAAGAGTTAATATA
+
AAAAA////E////E///E///E/E/////E/<//A///<E/////////E////A///E//E/E///////EE//E///
@NS500475:188:HW5KCBGXY:1:11101:21021:7600:TCGTATAGTCA 2:N:0:1
GGGTTTATTATTTTTTTGGCTTATCTTTTTCCAACAATGAATGAAGTCAGGGGCGGAACATAAGTAGAGACCATGGAGGA
+
AAAA/</EE/EEEEEEAE/A/EA/A/A////////E/E/A//A////A///<//AE////////E//EE/AE//E</E//
@NS500475:188:HW5KCBGXY:1:11101:24772:7600:CTTGGCGCTCC 2:N:0:1
AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
+
A//AAA/EEAEEE/AEEEEEEEEEEEEEEEEEE/EEEEEEEEEE/EE/A////AAE6//AEE/A/A////E/<///////
@NS500475:188:HW5KCBGXY:1:11101:10258:7600:GGTATGCAAGC 2:N:0:1
ACACCCGACTGCTCTTCCGAAGGTCCGAAGTTCAAATCCCAGCAATCACATGGTGGTTCACAACCATCCATAATGAGAT
+
AAAAAAEEEAEEA<<E<EE/AEEE/EEAEE/EE//EEEEEEEEE//EEE/EE/EEE/E/E</EEE/EAEEE/EEE/A/<
```

and run the command from above. @peterjc suggested in https://github.com/conda-forge/biopython-feedstock/issues/6 that Python3 could be the issue.


Discussion:
Confirmed, Python 3 issue - works fine on Python 2:

```
$ /usr/bin/python Bio/bgzf.py < Tests/Quality/example.fastq > /tmp/example.fastq.bgz
Producing BGZF output from stdin...
BGZF data produced
```

```
$ python3 Bio/bgzf.py < Tests/Quality/example.fastq > /tmp/example.fastq.bgz
Producing BGZF output from stdin...
Traceback (most recent call last):
  File "Bio/bgzf.py", line 871, in <module>
    w.close()
  File "Bio/bgzf.py", line 815, in close
    self.flush()
  File "Bio/bgzf.py", line 802, in flush
    self._write_block(self._buffer)
  File "Bio/bgzf.py", line 778, in _write_block
    self._handle.write(data)
TypeError: write() argument must be str, not bytes
```

In any case, using ``bgzf.py`` in this was almost an afterthought - I really only wrote it for testing. The ``bgzip`` from samtools/htslib would be a far more portable choice.
I think any fix will be some messy if statements to access ``sys.stdin.buffer`` and ``sys.stderr.buffer`` for the bytes handle, as per https://docs.python.org/3/library/sys.html

> To write or read binary data from/to the standard streams, use the underlying binary buffer object. For example, to write bytes to stdout, use sys.stdout.buffer.write(b'abc').

Resolving patch:
diff --git a/Bio/bgzf.py b/Bio/bgzf.py
--- a/Bio/bgzf.py
+++ b/Bio/bgzf.py
@@ -860,10 +860,19 @@ def __exit__(self, type, value, traceback):
         print("See also the tool bgzip that comes with samtools")
         sys.exit(0)
 
+    # Ensure we have binary mode handles
+    # (leave stderr as default text mode)
+    if sys.version_info[0] >= 3:
+        stdin = sys.stdin.buffer
+        stdout = sys.stdout.buffer
+    else:
+        stdin = sys.stdin
+        stdout = sys.stdout
+
     sys.stderr.write("Producing BGZF output from stdin...\n")
-    w = BgzfWriter(fileobj=sys.stdout)
+    w = BgzfWriter(fileobj=stdout)
     while True:
-        data = sys.stdin.read(65536)
+        data = stdin.read(65536)
         w.write(data)
         if not data:
             break


Test patch: