Repository: getmoto/moto

Issue:
moto should not take over `responses` defaults
We are using moto to mock AWS services (namely S3); and responses to control our access to third party APIs.

However, moto unconditionnally configures a global pass-through in `responses`:
At import time, `moto` configures responses:
https://github.com/spulec/moto/blob/master/moto/core/models.py#L249-L252

Once moto has been imported, it is no longer possible to use `responses` in one's code.

Since moto uses `@mock_XXX` idioms, it should only apply its configuration on code that is wrapped by such decorators/context managers.


Discussion:
Agreed @rbarrois, thanks for raising this. Will mark it as an enhancement
I agree with this. I don't know that it will be possible with the current `responses` API though. I think this would require us to add some features there to be able to remove pass-throughs and such.
I think we should at least document this behaviour somewhere and suggest a way to remove the hook. I currently use a hack way to remove the hook globally.

```python
import responses

responses._default_mock.add_passthru  = tuple()
responses..add_passthru = responses._default_mock.add_passthru
```

This should be done where `moto` is imported, most likely this would be in the `conftest.py`.

...but this solution is not optimal (imo). Maybe we should turn the logic around, so you have to explicitly enable the passthru if you really need this as most test suites should work without it anyway. With this the documentation would state, that this project uses responses, which has the side effect that all external requests are blocked and if external requests are needed they can simply be enabled by explicitly using `responses.add_passthru(SOME_URL_PREFIX)`.


Resolving patch:
diff --git a/moto/apigateway/models.py b/moto/apigateway/models.py
--- a/moto/apigateway/models.py
+++ b/moto/apigateway/models.py
@@ -41,6 +41,7 @@
     RestAPINotFound,
     ModelNotFound,
 )
+from ..core.models import responses_mock
 
 STAGE_URL = "https://{api_id}.execute-api.{region_name}.amazonaws.com/{stage_name}"
 
@@ -565,7 +566,7 @@ def update_integration_mocks(self, stage_name):
         )
 
         for url in [stage_url_lower, stage_url_upper]:
-            responses._default_mock._matches.insert(
+            responses_mock._matches.insert(
                 0,
                 responses.CallbackResponse(
                     url=url,
diff --git a/moto/core/models.py b/moto/core/models.py
--- a/moto/core/models.py
+++ b/moto/core/models.py
@@ -248,7 +248,7 @@ def _url_matches(self, url, other, match_querystring=False):
     target="botocore.vendored.requests.adapters.HTTPAdapter.send",
 )
 
-responses_mock = responses._default_mock
+responses_mock = responses.RequestsMock(assert_all_requests_are_fired=False)
 # Add passthrough to allow any other requests to work
 # Since this uses .startswith, it applies to http and https requests.
 responses_mock.add_passthru("http")


Test patch:
diff --git a/tests/test_apigateway/test_apigateway.py b/tests/test_apigateway/test_apigateway.py
--- a/tests/test_apigateway/test_apigateway.py
+++ b/tests/test_apigateway/test_apigateway.py
@@ -8,9 +8,9 @@
 import sure  # noqa
 from botocore.exceptions import ClientError
 
-import responses
 from moto import mock_apigateway, mock_cognitoidp, settings
 from moto.core import ACCOUNT_ID
+from moto.core.models import responses_mock
 import pytest
 
 
@@ -1786,8 +1786,8 @@ def test_get_model_with_invalid_name():
 
 @mock_apigateway
 def test_http_proxying_integration():
-    responses.add(
-        responses.GET, "http://httpbin.org/robots.txt", body="a fake response"
+    responses_mock.add(
+        responses_mock.GET, "http://httpbin.org/robots.txt", body="a fake response"
     )
 
     region_name = "us-west-2"
diff --git a/tests/test_sns/test_publishing_boto3.py b/tests/test_sns/test_publishing_boto3.py
--- a/tests/test_sns/test_publishing_boto3.py
+++ b/tests/test_sns/test_publishing_boto3.py
@@ -8,11 +8,11 @@
 from freezegun import freeze_time
 import sure  # noqa
 
-import responses
 from botocore.exceptions import ClientError
 import pytest
 from moto import mock_sns, mock_sqs, settings
 from moto.core import ACCOUNT_ID
+from moto.core.models import responses_mock
 from moto.sns import sns_backend
 
 MESSAGE_FROM_SQS_TEMPLATE = (
@@ -332,7 +332,7 @@ def callback(request):
         json.loads.when.called_with(request.body.decode()).should_not.throw(Exception)
         return 200, {}, ""
 
-    responses.add_callback(
+    responses_mock.add_callback(
         method="POST", url="http://example.com/foobar", callback=callback
     )