Repository: RaRe-Technologies/smart_open (python)

Issue:
Make compression_wrapper accept filename again
The `smart_open.compression.compression_wrapper()` used to accept a `filename` parameter. This is necessary in cases where the input `file_obj` cannot be modified = monkey-patchong a `.name` attribute into it is undesirable.

This feature was removed in 930c897aab96821bd5fff9cb6b0a2f8ab922f51e – @mpenkov do you remember why?

I propose bringing it back, because the alternative is hacky: accessing the protected `compression._COMPRESSOR_REGISTRY` dict directly.

Relatedly, the extension-matching logic is case-sensitive, `.gz` != `.GZ` etc. I don't see much point to this – should be case-insensitive IMO.

Pull request:
Bring back compression_wrapper(filename) + use case-insensitive extension matching
Fixes #736.

- [x] Add test(s), both for the "case-insensitive extension" and for `filename=X` itself.

Patch:
diff --git a/smart_open/compression.py b/smart_open/compression.py
index fe02246..08469e7 100644
--- a/smart_open/compression.py
+++ b/smart_open/compression.py
@@ -62,6 +62,7 @@ def register_compressor(ext, callback):
     """
     if not (ext and ext[0] == '.'):
         raise ValueError('ext must be a string starting with ., not %r' % ext)
+    ext = ext.lower()
     if ext in _COMPRESSOR_REGISTRY:
         logger.warning('overriding existing compression handler for %r', ext)
     _COMPRESSOR_REGISTRY[ext] = callback
@@ -103,24 +104,23 @@ def _handle_gzip(file_obj, mode):
     return result
 
 
-def compression_wrapper(file_obj, mode, compression):
+def compression_wrapper(file_obj, mode, compression=INFER_FROM_EXTENSION, filename=None):
     """
-    This function will wrap the file_obj with an appropriate
-    [de]compression mechanism based on the specified extension.
+    Wrap `file_obj` with an appropriate [de]compression mechanism based on its file extension.
 
-    file_obj must either be a filehandle object, or a class which behaves
-    like one. It must have a .name attribute.
+    If the filename extension isn't recognized, simply return the original `file_obj` unchanged.
 
-    If the filename extension isn't recognized, will simply return the original
-    file_obj.
+    `file_obj` must either be a filehandle object, or a class which behaves like one.
+
+    If `filename` is specified, it will be used to extract the extension.
+    If not, the `file_obj.name` attribute is used as the filename.
 
     """
     if compression == NO_COMPRESSION:
         return file_obj
     elif compression == INFER_FROM_EXTENSION:
         try:
-            filename = file_obj.name
-            filename.upper()  # make sure this thing is a string
+            filename = (filename or file_obj.name).lower()
         except (AttributeError, TypeError):
             logger.warning(
                 'unable to transparently decompress %r because it '


Test patch:
diff --git a/smart_open/tests/test_compression.py b/smart_open/tests/test_compression.py
new file mode 100644
index 0000000..4b93f63
--- /dev/null
+++ b/smart_open/tests/test_compression.py
@@ -0,0 +1,39 @@
+# -*- coding: utf-8 -*-
+#
+# Copyright (C) 2020 Radim Rehurek <me@radimrehurek.com>
+#
+# This code is distributed under the terms and conditions
+# from the MIT License (MIT).
+#
+import io
+import gzip
+import pytest
+
+import smart_open.compression
+
+
+plain = 'доброе утро планета!'.encode()
+
+
+def label(thing, name):
+    setattr(thing, 'name', name)
+    return thing
+
+
+@pytest.mark.parametrize(
+    'fileobj,compression,filename',
+    [
+        (io.BytesIO(plain), 'disable', None),
+        (io.BytesIO(plain), 'disable', ''),
+        (io.BytesIO(plain), 'infer_from_extension', 'file.txt'),
+        (io.BytesIO(plain), 'infer_from_extension', 'file.TXT'),
+        (io.BytesIO(plain), '.unknown', ''),
+        (io.BytesIO(gzip.compress(plain)), 'infer_from_extension', 'file.gz'),
+        (io.BytesIO(gzip.compress(plain)), 'infer_from_extension', 'file.GZ'),
+        (label(io.BytesIO(gzip.compress(plain)), 'file.gz'), 'infer_from_extension', ''),
+        (io.BytesIO(gzip.compress(plain)), '.gz', 'file.gz'),
+    ]
+)
+def test_compression_wrapper_read(fileobj, compression, filename):
+    wrapped = smart_open.compression.compression_wrapper(fileobj, 'rb', compression, filename)
+    assert wrapped.read() == plain