Repository: getsentry/responses
Pull request #665 (2023-08-20 15:22:18)

Support gzipped request.body in json_params_matcher
I just discovered `matchers` (absolute lifesaver in concurrent tests), but I currently need to disable gzip for the tests since it isn't supported ootb.

Ltmk if there are additional changed/updates/changelog entries etc. you want me to create 😄 

codecov[bot]: ## [Codecov](https://app.codecov.io/gh/getsentry/responses/pull/665?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=getsentry) Report
Patch coverage: **`100.00%`** and no project coverage change.
> Comparison is base [(`ad22f54`)](https://app.codecov.io/gh/getsentry/responses/commit/ad22f54e941b95b8cde8c03e0b20a6dd1ffb4dde?el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=getsentry) 100.00% compared to head [(`ee68266`)](https://app.codecov.io/gh/getsentry/responses/pull/665?src=pr&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=getsentry) 100.00%.

<details><summary>Additional details and impacted files</summary>


```diff
@@            Coverage Diff            @@
##            master      #665   +/-   ##
=========================================
  Coverage   100.00%   100.00%           
=========================================
  Files            9         9           
  Lines         2880      2891   +11     
=========================================
+ Hits          2880      2891   +11     
```


| [Files Changed](https://app.codecov.io/gh/getsentry/responses/pull/665?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=getsentry) | Coverage Δ | |
|---|---|---|
| [responses/matchers.py](https://app.codecov.io/gh/getsentry/responses/pull/665?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=getsentry#diff-cmVzcG9uc2VzL21hdGNoZXJzLnB5) | `100.00% <100.00%> (ø)` | |
| [responses/tests/test\_matchers.py](https://app.codecov.io/gh/getsentry/responses/pull/665?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=getsentry#diff-cmVzcG9uc2VzL3Rlc3RzL3Rlc3RfbWF0Y2hlcnMucHk=) | `100.00% <100.00%> (ø)` | |


</details>

[:umbrella: View full report in Codecov by Sentry](https://app.codecov.io/gh/getsentry/responses/pull/665?src=pr&el=continue&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=getsentry).   
:loudspeaker: Have feedback on the report? [Share it here](https://about.codecov.io/codecov-pr-comment-feedback/?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=getsentry).

markstory: Looking good. There is an error from mypy that will need to be resolved before this can be merged.

> responses/matchers.py:132:52: error: Argument 1 to "decompress" has incompatible type "bytes | str"; expected "Buffer"  [arg-type]
haakonvt: > Looking good. There is an error from mypy that will need to be resolved before this can be merged.
> 
> > responses/matchers.py:132:52: error: Argument 1 to "decompress" has incompatible type "bytes | str"; expected "Buffer"  [arg-type]

That one surprised me, seems like mypy think the error can be raised after variable has been updated... I wasn't sure how you feel about `cast` in this codebase, so I changed a few `request_body` to `request.body` to help `mypy` understand.

Patch:
diff --git a/.pre-commit-config.yaml b/.pre-commit-config.yaml
index f7e1f83..899de43 100644
--- a/.pre-commit-config.yaml
+++ b/.pre-commit-config.yaml
@@ -25,7 +25,12 @@ repos:
     hooks:
       - id: flake8
         args: [ '--max-line-length', '100', '--max-doc-length', '120' ]
-  - repo: https://github.com/asottile/blacken-docs
+  - repo: https://github.com/asottile/pyupgrade
+    rev: v3.10.1
+    hooks:
+    -   id: pyupgrade
+        args: [--py37-plus]
+  - repo: https://github.com/adamchainz/blacken-docs
     rev: 1.14.0
     hooks:
       - id: blacken-docs
diff --git a/responses/__init__.py b/responses/__init__.py
index 2666b7c..a89e2d7 100644
--- a/responses/__init__.py
+++ b/responses/__init__.py
@@ -111,8 +111,6 @@ class FalseBool:
     def __bool__(self) -> bool:
         return False
 
-    __nonzero__ = __bool__
-
 
 def urlencoded_params_matcher(params: Optional[Dict[str, str]]) -> Callable[..., Any]:
     warn(
@@ -369,7 +367,7 @@ def _handle_body(
     return data
 
 
-class BaseResponse(object):
+class BaseResponse:
     passthrough: bool = False
     content_type: Optional[str] = None
     headers: Optional[Mapping[str, str]] = None
@@ -662,7 +660,7 @@ class PassthroughResponse(BaseResponse):
         super().__init__(*args, passthrough=True, **kwargs)
 
 
-class RequestsMock(object):
+class RequestsMock:
     DELETE: Literal["DELETE"] = "DELETE"
     GET: Literal["GET"] = "GET"
     HEAD: Literal["HEAD"] = "HEAD"
@@ -792,7 +790,7 @@ class RequestsMock(object):
     def _parse_response_file(
         self, file_path: "Union[str, bytes, os.PathLike[Any]]"
     ) -> "Dict[str, Any]":
-        with open(file_path, "r") as file:
+        with open(file_path) as file:
             data = yaml.safe_load(file)
         return data
 
@@ -1047,7 +1045,7 @@ class RequestsMock(object):
             if self.passthru_prefixes:
                 error_msg += "Passthru prefixes:\n"
                 for p in self.passthru_prefixes:
-                    error_msg += "- {}\n".format(p)
+                    error_msg += f"- {p}\n"
 
             response = ConnectionError(error_msg)
             response.request = request
@@ -1156,7 +1154,7 @@ class RequestsMock(object):
         not_called = [m for m in self.registered() if m.call_count == 0]
         if not_called:
             raise AssertionError(
-                "Not all requests have been executed {0!r}".format(
+                "Not all requests have been executed {!r}".format(
                     [(match.method, match.url) for match in not_called]
                 )
             )
diff --git a/responses/matchers.py b/responses/matchers.py
index e9fb372..43773cb 100644
--- a/responses/matchers.py
+++ b/responses/matchers.py
@@ -1,3 +1,4 @@
+import gzip
 import json as json_module
 import re
 from json.decoder import JSONDecodeError
@@ -48,7 +49,7 @@ def _create_key_val_str(input_dict: Union[Dict[Any, Any], Any]) -> str:
         elif isinstance(val, list):
             val = list_to_str(input_list=val)
 
-        items_list.append("{}: {}".format(key, val))
+        items_list.append(f"{key}: {val}")
 
     key_val_str = "{{{}}}".format(", ".join(items_list))
     return key_val_str
@@ -125,8 +126,11 @@ def json_params_matcher(
         request_body = request.body
         json_params = (params or {}) if not isinstance(params, list) else params
         try:
-            if isinstance(request_body, bytes):
-                request_body = request_body.decode("utf-8")
+            if isinstance(request.body, bytes):
+                try:
+                    request_body = request.body.decode("utf-8")
+                except UnicodeDecodeError:
+                    request_body = gzip.decompress(request.body).decode("utf-8")
             json_body = json_module.loads(request_body) if request_body else {}
 
             if (
diff --git a/responses/registries.py b/responses/registries.py
index 4b7e233..fcf481a 100644
--- a/responses/registries.py
+++ b/responses/registries.py
@@ -11,7 +11,7 @@ if TYPE_CHECKING:  # pragma: no cover
     from responses import BaseResponse
 
 
-class FirstMatchRegistry(object):
+class FirstMatchRegistry:
     def __init__(self) -> None:
         self._responses: List["BaseResponse"] = []
 
@@ -67,9 +67,7 @@ class FirstMatchRegistry(object):
         try:
             index = self.registered.index(response)
         except ValueError:
-            raise ValueError(
-                "Response is not registered for URL {}".format(response.url)
-            )
+            raise ValueError(f"Response is not registered for URL {response.url}")
         self.registered[index] = response
         return response
 


Test patch:
diff --git a/responses/tests/test_matchers.py b/responses/tests/test_matchers.py
index 237b435..70ee71a 100644
--- a/responses/tests/test_matchers.py
+++ b/responses/tests/test_matchers.py
@@ -1,3 +1,4 @@
+import gzip
 import re
 from unittest.mock import Mock
 
@@ -158,6 +159,14 @@ def test_json_params_matcher_json_list_empty():
     assert result == (True, "")
 
 
+def test_json_params_matcher_body_is_gzipped():
+    json_a = {"foo": 42, "bar": None}
+    json_b = gzip.compress('{"foo": 42, "bar": null}'.encode("utf-8"))
+    mock_request = Mock(body=json_b)
+    result = matchers.json_params_matcher(json_a)(mock_request)
+    assert result == (True, "")
+
+
 def test_urlencoded_params_matcher_blank():
     @responses.activate
     def run():
diff --git a/responses/tests/test_recorder.py b/responses/tests/test_recorder.py
index bcd05ce..d892845 100644
--- a/responses/tests/test_recorder.py
+++ b/responses/tests/test_recorder.py
@@ -87,7 +87,7 @@ class TestRecord:
 
         run()
 
-        with open(self.out_file, "r") as file:
+        with open(self.out_file) as file:
             data = yaml.safe_load(file)
 
         assert data == get_data(httpserver.host, httpserver.port)
diff --git a/responses/tests/test_responses.py b/responses/tests/test_responses.py
index cbcee54..2ae3d65 100644
--- a/responses/tests/test_responses.py
+++ b/responses/tests/test_responses.py
@@ -1,5 +1,3 @@
-# coding: utf-8
-
 import inspect
 import os
 import re
@@ -912,7 +910,7 @@ def fruit_basket(my_fruit):
 
 
 @pytest.mark.usefixtures("my_fruit", "fruit_basket")
-class TestFixtures(object):
+class TestFixtures:
     """
     Test that pytest fixtures work well with 'activate' decorator
     """
@@ -960,7 +958,7 @@ def test_activate_doesnt_change_signature_with_return_type():
 
 
 def test_activate_doesnt_change_signature_for_method():
-    class TestCase(object):
+    class TestCase:
         def test_function(self, a, b=None):
             return self, a, b
 
@@ -1100,10 +1098,10 @@ def test_response_filebody():
     def run():
         current_file = os.path.abspath(__file__)
         with responses.RequestsMock() as m:
-            with open(current_file, "r", encoding="utf-8") as out:
+            with open(current_file, encoding="utf-8") as out:
                 m.add(responses.GET, "http://example.com", body=out.read(), stream=True)
                 resp = requests.get("http://example.com", stream=True)
-            with open(current_file, "r", encoding="utf-8") as out:
+            with open(current_file, encoding="utf-8") as out:
                 assert resp.text == out.read()
 
     run()
@@ -1195,7 +1193,7 @@ def test_assert_all_requests_fired_multiple():
 def test_allow_redirects_samehost():
     redirecting_url = "http://example.com"
     final_url_path = "/1"
-    final_url = "{0}{1}".format(redirecting_url, final_url_path)
+    final_url = f"{redirecting_url}{final_url_path}"
     url_re = re.compile(r"^http://example.com(/)?(\d+)?$")
 
     def request_callback(request):
@@ -1209,7 +1207,7 @@ def test_allow_redirects_samehost():
                 n = 1
             else:
                 n = 0
-            redirect_headers = {"location": "/{0!s}".format(n)}
+            redirect_headers = {"location": f"/{n!s}"}
             return 301, redirect_headers, None
 
     def run():
@@ -1696,12 +1694,12 @@ class TestPassthru:
         @responses.activate
         def run():
             responses.add(PassthroughResponse(responses.GET, url))
-            responses.add(responses.GET, "{}/one".format(url), body="one")
+            responses.add(responses.GET, f"{url}/one", body="one")
             responses.add(responses.GET, "http://example.com/two", body="two")
 
             resp = requests.get("http://example.com/two")
             assert_response(resp, "two")
-            resp = requests.get("{}/one".format(url))
+            resp = requests.get(f"{url}/one")
             assert_response(resp, "one")
             resp = requests.get(url)
             assert_response(resp, "OK")
@@ -1762,12 +1760,12 @@ class TestPassthru:
         @responses.activate
         def run():
             responses.add_passthru(url)
-            responses.add(responses.GET, "{}/one".format(url), body="one")
+            responses.add(responses.GET, f"{url}/one", body="one")
             responses.add(responses.GET, "http://example.com/two", body="two")
 
             resp = requests.get("http://example.com/two")
             assert_response(resp, "two")
-            resp = requests.get("{}/one".format(url))
+            resp = requests.get(f"{url}/one")
             assert_response(resp, "one")
             resp = requests.get(url)
             assert_response(resp, "OK")
@@ -1784,7 +1782,7 @@ class TestPassthru:
         @responses.activate
         def run():
             responses.add_passthru(re.compile(f"{url}/\\w+"))
-            responses.add(responses.GET, "{}/one".format(url), body="one")
+            responses.add(responses.GET, f"{url}/one", body="one")
             responses.add(responses.GET, "http://example.com/two", body="two")
 
             resp = requests.get("http://example.com/two")
@@ -1876,7 +1874,7 @@ def test_request_param(url):
         params = {"hello": "world", "example": "params"}
         responses.add(
             method=responses.GET,
-            url="{0}?hello=world".format(url),
+            url=f"{url}?hello=world",
             body="test",
             match_querystring=False,
         )
@@ -1923,7 +1921,7 @@ def test_assert_call_count(url):
 
         with pytest.raises(AssertionError) as excinfo:
             responses.assert_call_count(url, 2)
-        assert "Expected URL '{0}' to be called 2 times. Called 0 times.".format(
+        assert "Expected URL '{}' to be called 2 times. Called 0 times.".format(
             url
         ) in str(excinfo.value)
 
@@ -1936,7 +1934,7 @@ def test_assert_call_count(url):
         requests.get(url)
         with pytest.raises(AssertionError) as excinfo:
             responses.assert_call_count(url, 3)
-        assert "Expected URL '{0}' to be called 3 times. Called 2 times.".format(
+        assert "Expected URL '{}' to be called 3 times. Called 2 times.".format(
             url
         ) in str(excinfo.value)