Repository: biopython/biopython

Issue:
BgzfReader argument `mode` not tested if a fileobj is specified
`BgzfReader.__init__()` has an argument mode, which is thoroughly tested, but only if a filename is specified, not in the case of a fileobj.

https://github.com/biopython/biopython/blob/207af955f21628f0b0e002f9c86ef2eefaf7ef14/Bio/bgzf.py#L558-L583

The placing of the `mode` argument testing in the else code block seems to indicate it has something to do with opening the file, but it doesn't, but the error message does. The word variable name mode here apparently confused the programmer and maybe a "content_mode" instead of mode for the treatment of the file content would remove some of the confusion. The mode for opening is hardcoded as 'rb'. Also the mode/content_mode is not well documented for users. Working the comments into a docstring would solve that. 

I believe that the use of the mode argument might have changed over time, because it contains the "r" in the BgzfReader docstring which has no effect whatsoever in the current code.


Discussion:
It controls if we are in text mode or bytes mode (line 579 as quoted):

```python
self._text = "b" not in mode.lower() 
```

The internal handle to the compressed file itself is always in binary (bytes) mode.
I agree whit what you say. The initial Title and isse was a reflection of my misunderstanding of the internal workings of BgzfReader, but the issue now raised above are real. (If you read the issue text in the email, that was my misunderstanding and you seem to answer to that. I am sorry for the confusion.) Don't you agree? 
I disagree. New title says "BgzfReader argument mode not tested if a fileobj is specified", but it is checked:

```python
        if fileobj:
            if "b" not in fileobj.mode.lower():
                raise ValueError("fileobj not opened in binary mode")
            handle = fileobj
```

Internally the compressed data handle must be binary. So if given a file object we check is it binary, but if given a file name we open that as binary.

The mode argument to ``__init__`` is only for controlling the datatype of the decompressed data we are going to  provide, i.e. this controls unicode strings (text mode) versus bytes (binary mode). Note this class only offers input, not output, which is the other thing a file mode argument usually deals with.

The function might benefit from having a docstring which discusses this? Do you feel you could write that now?
Please see the history: mode was once used in open, but when that use changed, the code didn't change consistently throughout the `__init__` function. Some context here: https://github.com/fabianegli/biopython/blame/791f9803892536fc643191ded7bb20fd61606ae9/Bio/bgzf.py

At this point https://github.com/fabianegli/biopython/blame/791f9803892536fc643191ded7bb20fd61606ae9/Bio/bgzf.py#L459
it made sense to check for "a" and "w" in the mode because mode was used for the file opening.  But it is not used as such any more. So these tests there are obsolete (and confusing).
> I disagree. New title says "BgzfReader argument mode not tested if a fileobj is specified", but it is checked:

This is exactly my issue: when you say the mode is checked, you think of the mode of the fileobj, but what I am talking about is that the `mode` argument is not checked. It is this confusion of the mode for file IO and mode of file content that makes it difficult to concisely discuss the issue but also led to the code being buggy.
Are you saying this should be run regardless of being given filename or fileobj?

```python
         if "w" in mode.lower() or "a" in mode.lower(): 
             raise ValueError( 
                 "Must use read mode (default), not write or append mode" 
             ) 
```

That looks like a sensible improvement (not tested).
No, I think this test is completely obsolete now. Mode isn't used for file opening any more. This test should be removed completely.

Another fact that adds to the confusion is that the function definition sets `mode="r"`. But the only thing that matters is whether the `mode` argument contains a "b" or not when the `_text` attribute is computed.
Yes, right now the main thing that matters is if the ``mode`` argument contains ``"b"`` or not.

However, asking for write with ``mode='w'`` or ``mode="wb"``, or append with ``mode='a'`` or ``mode="ab"`` etc should fail explicitly (and not silently be treated as read mode).

Also, cross reference #2166 for getting append mode to work.
A different test would be in order for mode. It is only a boolean choice, but the argument accepts a string that resembles a mode.

Options: 

* Change the argument name to something like content_mode_binary and make it boolean.
* Keep the name but only accept "b" or "latin-1", because that's what it is decoded as.
No. It deliberately follows the Python ``open(...)`` and ``gzip.open(...)`` function argument name.

Right now it only needs to accept "r", "rt", or "rb", so that validation test could be turned round (instead of complaining if it sees "a" or "w").
> It deliberately follows the Python open(...) and gzip.open(...) function argument name.

This is where I disagree, because the mode argument of the reader is not used the same way as in the cited open() functions and has a different meaning.

There is a discrepancy between how the mode argument is used in the reader and writer object. While the reader only uses it to know the content mode, the writer selects the file io mode with it.
> Right now it only needs to accept "r", "rt", or "rb", so that validation test could be turned round (instead of complaining if it sees "a" or "w").

It does not test for that and neither does "r" or "t" change the behavior of the reader's code. You can use it with even weirder mode, e.g.

````python
>>> bgzf.BgzfReader("./Tests/Quality/example.fastq.bgz", "xyz")
<Bio.bgzf.BgzfReader object at 0x10751ca60>
```python
>>> bgzf.BgzfReader(fileobj=open("./Tests/Quality/example.fastq.bgz", "rb"), mode="ra")
<Bio.bgzf.BgzfReader object at 0x1080779d0>
>>> bgzf.BgzfReader(filename="./Tests/Quality/example.fastq.bgz", mode="ra")
...
ValueError: Must use read mode (default), not write or append mode
>>> open("./Tests/Quality/example.fastq.bgz", mode="ra")
...
ValueError: must have exactly one of create/read/write/append mode
```

The above code clearly shows that the same mode works for one file input but not the other. That shouldn't be the case.

The problem is that the mode argument is sometimes thought to handle file IO and sometimes the encoding. This is not just confusing, but bound to be erroneous because only one argument has to handle two switches, one for the encoding and one for file IO. This is neither easy to follow when reading code nor easy to understand as user nor do I think it is good practice to hide different code switches behind a single argument. What happens is that a choice unrelated to the mode argument in the builtin open (namely the encoding) needs to be added to the string of the mode argument, but not only that it is also encoded as "b" which by itself has a meaning in the builtin open mode (file IO as bytes) but a different one what it stands for when used in `bgzf.open` (latin-1 encoding or not). 
> However, asking for write with mode='w' or mode="wb", or append with mode='a' or mode="ab" etc should fail explicitly (and not silently be treated as read mode).

I agree that the mode (and generally any argument) needs to be well defined and the user input tested.
Your example shows way always running this snippet is worth doing for consistency:

```
         if "w" in mode.lower() or "a" in mode.lower(): 
             raise ValueError( 
                 "Must use read mode (default), not write or append mode" 
             ) 
```
See discussion about encodings and why it is hard coded to ``latin-1`` here:

https://github.com/biopython/biopython/blob/biopython-179/Bio/bgzf.py#L223
I haven't really followed this discussion, but wouldn't it be better to use `io.TextIOWrapper` as a wrapper to take care of the bytes-to-string conversion? This is the approach taken by `gzip` in the standard library. It seems odd to have things like `self._newline` hardcoded in `Bio.bgzf`.

Resolving patch:
diff --git a/Bio/bgzf.py b/Bio/bgzf.py
--- a/Bio/bgzf.py
+++ b/Bio/bgzf.py
@@ -556,7 +556,32 @@ class BgzfReader:
     """
 
     def __init__(self, filename=None, mode="r", fileobj=None, max_cache=100):
-        """Initialize the class."""
+        r"""Initialize the class for reading a BGZF file.
+
+        You would typically use the top level ``bgzf.open(...)`` function
+        which will call this class internally. Direct use is discouraged.
+
+        Either the ``filename`` (string) or ``fileobj`` (input file object in
+        binary mode) arguments must be supplied, but not both.
+
+        Argument ``mode`` controls if the data will be returned as strings in
+        text mode ("rt", "tr", or default "r"), or bytes binary mode ("rb"
+        or "br"). The argument name matches the built-in ``open(...)`` and
+        standard library ``gzip.open(...)`` function.
+
+        If text mode is requested, in order to avoid multi-byte characters,
+        this is hard coded to use the "latin1" encoding, and "\r" and "\n"
+        are passed as is (without implementing universal new line mode). There
+        is no ``encoding`` argument.
+
+        If your data is in UTF-8 or any other incompatible encoding, you must
+        use binary mode, and decode the appropriate fragments yourself.
+
+        Argument ``max_cache`` controls the maximum number of BGZF blocks to
+        cache in memory. Each can be up to 64kb thus the default of 100 blocks
+        could take up to 6MB of RAM. This is important for efficient random
+        access, a small value is fine for reading the file in one pass.
+        """
         # TODO - Assuming we can seek, check for 28 bytes EOF empty block
         # and if missing warn about possible truncation (as in samtools)?
         if max_cache < 1:
@@ -566,15 +591,16 @@ def __init__(self, filename=None, mode="r", fileobj=None, max_cache=100):
         # bytes under Python 3)
         if filename and fileobj:
             raise ValueError("Supply either filename or fileobj, not both")
+        # Want to reject output modes like w, a, x, +
+        if mode.lower() not in ("r", "tr", "rt", "rb", "br"):
+            raise ValueError(
+                "Must use a read mode like 'r' (default), 'rt', or 'rb' for binary"
+            )
         if fileobj:
             if "b" not in fileobj.mode.lower():
                 raise ValueError("fileobj not opened in binary mode")
             handle = fileobj
         else:
-            if "w" in mode.lower() or "a" in mode.lower():
-                raise ValueError(
-                    "Must use read mode (default), not write or append mode"
-                )
             handle = _open(filename, "rb")
         self._text = "b" not in mode.lower()
         if self._text:


Test patch: