Repository: boto/botocore
Pull request #2573 (2021-12-10 18:42:47)

Properly catch ProtocolError on streamed responses
This PR should address boto/boto3#2902 by properly encapsulating deferred ProtocolError's raised by urllib3 when streaming responses. The new error `ResponseStreamingError` inherits from `HTTPClientError` since it's still fundamentally a networking issue that's been masked until read time.

codecov-commenter: # [Codecov](https://codecov.io/gh/boto/botocore/pull/2573?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=boto) Report
> Merging [#2573](https://codecov.io/gh/boto/botocore/pull/2573?src=pr&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=boto) (129c32a) into [develop](https://codecov.io/gh/boto/botocore/commit/6d3a54f9039dbb5899cd09631ed7e29e90ce3adf?el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=boto) (6d3a54f) will **decrease** coverage by `0.14%`.
> The diff coverage is `100.00%`.

[![Impacted file tree graph](https://codecov.io/gh/boto/botocore/pull/2573/graphs/tree.svg?width=650&height=150&src=pr&token=QOLXw6XSza&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=boto)](https://codecov.io/gh/boto/botocore/pull/2573?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=boto)

```diff
@@             Coverage Diff             @@
##           develop    #2573      +/-   ##
===========================================
- Coverage    95.40%   95.26%   -0.15%     
===========================================
  Files           59       59              
  Lines        11618    11484     -134     
===========================================
- Hits         11084    10940     -144     
- Misses         534      544      +10     
```


| [Impacted Files](https://codecov.io/gh/boto/botocore/pull/2573?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=boto) | Coverage Δ | |
|---|---|---|
| [botocore/exceptions.py](https://codecov.io/gh/boto/botocore/pull/2573/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=boto#diff-Ym90b2NvcmUvZXhjZXB0aW9ucy5weQ==) | `99.52% <100.00%> (+<0.01%)` | :arrow_up: |
| [botocore/response.py](https://codecov.io/gh/boto/botocore/pull/2573/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=boto#diff-Ym90b2NvcmUvcmVzcG9uc2UucHk=) | `92.95% <100.00%> (+0.31%)` | :arrow_up: |
| [botocore/compat.py](https://codecov.io/gh/boto/botocore/pull/2573/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=boto#diff-Ym90b2NvcmUvY29tcGF0LnB5) | `71.82% <0.00%> (-1.41%)` | :arrow_down: |
| [botocore/httpsession.py](https://codecov.io/gh/boto/botocore/pull/2573/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=boto#diff-Ym90b2NvcmUvaHR0cHNlc3Npb24ucHk=) | `89.27% <0.00%> (-1.37%)` | :arrow_down: |
| [botocore/hooks.py](https://codecov.io/gh/boto/botocore/pull/2573/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=boto#diff-Ym90b2NvcmUvaG9va3MucHk=) | `95.27% <0.00%> (-0.40%)` | :arrow_down: |
| [botocore/utils.py](https://codecov.io/gh/boto/botocore/pull/2573/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=boto#diff-Ym90b2NvcmUvdXRpbHMucHk=) | `95.40% <0.00%> (-0.35%)` | :arrow_down: |
| [botocore/model.py](https://codecov.io/gh/boto/botocore/pull/2573/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=boto#diff-Ym90b2NvcmUvbW9kZWwucHk=) | `98.03% <0.00%> (-0.23%)` | :arrow_down: |
| [botocore/docs/bcdoc/restdoc.py](https://codecov.io/gh/boto/botocore/pull/2573/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=boto#diff-Ym90b2NvcmUvZG9jcy9iY2RvYy9yZXN0ZG9jLnB5) | `96.26% <0.00%> (-0.17%)` | :arrow_down: |
| [botocore/retries/bucket.py](https://codecov.io/gh/boto/botocore/pull/2573/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=boto#diff-Ym90b2NvcmUvcmV0cmllcy9idWNrZXQucHk=) | `98.48% <0.00%> (-0.09%)` | :arrow_down: |
| [botocore/loaders.py](https://codecov.io/gh/boto/botocore/pull/2573/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=boto#diff-Ym90b2NvcmUvbG9hZGVycy5weQ==) | `99.24% <0.00%> (-0.04%)` | :arrow_down: |
| ... and [14 more](https://codecov.io/gh/boto/botocore/pull/2573/diff?src=pr&el=tree-more&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=boto) | |

------

[Continue to review full report at Codecov](https://codecov.io/gh/boto/botocore/pull/2573?src=pr&el=continue&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=boto).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=boto)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/boto/botocore/pull/2573?src=pr&el=footer&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=boto). Last update [6d3a54f...129c32a](https://codecov.io/gh/boto/botocore/pull/2573?src=pr&el=lastupdated&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=boto). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=boto).


Patch:
diff --git a/botocore/exceptions.py b/botocore/exceptions.py
index f2fbcbc71..65784270c 100644
--- a/botocore/exceptions.py
+++ b/botocore/exceptions.py
@@ -142,6 +142,10 @@ class ProxyConnectionError(ConnectionError, requests.exceptions.ProxyError):
     fmt = 'Failed to connect to proxy URL: "{proxy_url}"'
 
 
+class ResponseStreamingError(HTTPClientError):
+    fmt = 'An error occurred while reading from response stream: {error}'
+
+
 class NoCredentialsError(BotoCoreError):
     """
     No credentials could be found.
diff --git a/botocore/response.py b/botocore/response.py
index be32345c4..a381a06c5 100644
--- a/botocore/response.py
+++ b/botocore/response.py
@@ -15,8 +15,11 @@
 import logging
 
 from botocore.compat import set_socket_timeout
-from botocore.exceptions import IncompleteReadError, ReadTimeoutError
+from botocore.exceptions import (
+    IncompleteReadError, ReadTimeoutError, ResponseStreamingError
+)
 from urllib3.exceptions import ReadTimeoutError as URLLib3ReadTimeoutError
+from urllib3.exceptions import ProtocolError as URLLib3ProtocolError
 from botocore import parsers
 
 # Keep these imported.  There's pre-existing code that uses them.
@@ -80,6 +83,8 @@ class StreamingBody(object):
         except URLLib3ReadTimeoutError as e:
             # TODO: the url will be None as urllib3 isn't setting it yet
             raise ReadTimeoutError(endpoint_url=e.url, error=e)
+        except URLLib3ProtocolError as e:
+            raise ResponseStreamingError(error=e)
         self._amount_read += len(chunk)
         if amt is None or (not chunk and amt > 0):
             # If the server sends empty contents or


Test patch:
diff --git a/tests/unit/test_response.py b/tests/unit/test_response.py
index 29afcd0be..e7ccf8cf5 100644
--- a/tests/unit/test_response.py
+++ b/tests/unit/test_response.py
@@ -16,11 +16,14 @@ import datetime
 
 from dateutil.tz import tzutc
 from urllib3.exceptions import ReadTimeoutError as URLLib3ReadTimeoutError
+from urllib3.exceptions import ProtocolError as URLLib3ProtocolError
 
 import botocore
 from botocore import response
 from botocore.compat import six
-from botocore.exceptions import IncompleteReadError, ReadTimeoutError
+from botocore.exceptions import (
+    IncompleteReadError, ReadTimeoutError, ResponseStreamingError
+)
 from botocore.awsrequest import AWSResponse
 
 XMLBODY1 = (b'<?xml version="1.0" encoding="UTF-8"?><Error>'
@@ -166,6 +169,20 @@ class TestStreamWrapper(unittest.TestCase):
         with self.assertRaises(ReadTimeoutError):
             stream.read()
 
+    def test_catches_urllib3_protocol_error(self):
+        class ProtocolErrorBody:
+            def read(*args, **kwargs):
+                raise URLLib3ProtocolError(None, None, None)
+
+            def geturl(*args, **kwargs):
+                return 'http://example.com'
+
+        stream = response.StreamingBody(
+            ProtocolErrorBody(), content_length=None
+        )
+        with self.assertRaises(ResponseStreamingError):
+            stream.read()
+
     def test_streaming_line_abstruse_newline_standard(self):
         for chunk_size in range(1, 30):
             body = six.BytesIO(b'1234567890\r\n1234567890\r\n12345\r\n')