Repository: redis/redis-py

Issue:
Behaviour of 'encoding_errors' differs when hiredis-py is installed
**Version**: redis-py 3.2.1 installed using pip; redis 5.0.4 built from source; (optionally) hiredis-py 1.0.0 built from from source

**Platform**: python 3.6.7 on Ubuntu 18.04

**Description**:

When hiredis-py is not installed, `encoding_errors` behaves as expected:

```
root@redis:~# python3
Python 3.6.7 (default, Oct 22 2018, 11:32:17)
[GCC 8.2.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import redis
>>> r = redis.Redis(decode_responses=True, encoding_errors="replace")
>>> r.set("foo",b"bar\xff")
True
>>> r.get("foo")
'bar�'
>>> r = redis.Redis(decode_responses=True, encoding_errors="ignore")
>>> r.get("foo")
'bar'
>>>
```

However, when hiredis-py is available, it behaves differently, and the `encoding_errors` option doesn't appear to make any difference:

```
>>> r = redis.Redis(decode_responses=True, encoding_errors="replace")
>>> r.get("foo")
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/usr/local/lib/python3.6/dist-packages/redis/client.py", line 1264, in get
    return self.execute_command('GET', name)
  File "/usr/local/lib/python3.6/dist-packages/redis/client.py", line 775, in execute_command
    return self.parse_response(connection, command_name, **options)
  File "/usr/local/lib/python3.6/dist-packages/redis/client.py", line 789, in parse_response
    response = connection.read_response()
  File "/usr/local/lib/python3.6/dist-packages/redis/connection.py", line 637, in read_response
    response = self._parser.read_response()
  File "/usr/local/lib/python3.6/dist-packages/redis/connection.py", line 414, in read_response
    response = self._reader.gets()
UnicodeDecodeError: 'utf-8' codec can't decode byte 0xff in position 3: invalid start byte
>>> r = redis.Redis(decode_responses=True, encoding_errors="ignore")
>>> r.get("foo")
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/usr/local/lib/python3.6/dist-packages/redis/client.py", line 1264, in get
    return self.execute_command('GET', name)
  File "/usr/local/lib/python3.6/dist-packages/redis/client.py", line 775, in execute_command
    return self.parse_response(connection, command_name, **options)
  File "/usr/local/lib/python3.6/dist-packages/redis/client.py", line 789, in parse_response
    response = connection.read_response()
  File "/usr/local/lib/python3.6/dist-packages/redis/connection.py", line 637, in read_response
    response = self._parser.read_response()
  File "/usr/local/lib/python3.6/dist-packages/redis/connection.py", line 414, in read_response
    response = self._reader.gets()
UnicodeDecodeError: 'utf-8' codec can't decode byte 0xff in position 3: invalid start byte
>>>
```

It appears that the `errors` option is not being passed to `hiredis.Reader`


Discussion:

Resolving patch:
diff --git a/redis/connection.py b/redis/connection.py
--- a/redis/connection.py
+++ b/redis/connection.py
@@ -41,6 +41,8 @@
         hiredis_version >= StrictVersion('0.1.3')
     HIREDIS_SUPPORTS_BYTE_BUFFER = \
         hiredis_version >= StrictVersion('0.1.4')
+    HIREDIS_SUPPORTS_ENCODING_ERRORS = \
+        hiredis_version >= StrictVersion('1.0.0')
 
     if not HIREDIS_SUPPORTS_BYTE_BUFFER:
         msg = ("redis-py works best with hiredis >= 0.1.4. You're running "
@@ -318,6 +320,8 @@ def on_connect(self, connection):
 
         if connection.encoder.decode_responses:
             kwargs['encoding'] = connection.encoder.encoding
+        if HIREDIS_SUPPORTS_ENCODING_ERRORS:
+            kwargs['errors'] = connection.encoder.encoding_errors
         self._reader = hiredis.Reader(**kwargs)
         self._next_response = False
 


Test patch:
diff --git a/tests/test_encoding.py b/tests/test_encoding.py
--- a/tests/test_encoding.py
+++ b/tests/test_encoding.py
@@ -25,6 +25,20 @@ def test_list_encoding(self, r):
         assert r.lrange('a', 0, -1) == result
 
 
+class TestEncodingErrors(object):
+    def test_ignore(self, request):
+        r = _get_client(redis.Redis, request=request, decode_responses=True,
+                        encoding_errors='ignore')
+        r.set('a', b'foo\xff')
+        assert r.get('a') == 'foo'
+
+    def test_replace(self, request):
+        r = _get_client(redis.Redis, request=request, decode_responses=True,
+                        encoding_errors='replace')
+        r.set('a', b'foo\xff')
+        assert r.get('a') == 'foo\ufffd'
+
+
 class TestCommandsAreNotEncoded(object):
     @pytest.fixture()
     def r(self, request):