Repository: jamielennox/requests-mock

Issue:
redirects not working with second url real_http=True
```
with requests_mock.mock() as req_mock:
    req_mock.post(url1, status_code=HTTPStatus.FOUND, headers={'location': url2})
    req_mock.get(url2, real_http=True)

requests.post(url1)
```

this is resulting ConnectionError (since url1 is not reachable)

from what I can tell the redirection works ok, than mocker._fake_send calling _original_send and adapter._RunRealHTTP is being raised and caught (which is ok), it then tries to restore the original requests session adapter but unfortunately because the first redirection has yet to finish, the original adapter was not restored, so what the second requests consider the original requests session adapter is actually the fake adapter

Discussion:
clslgrnc: Codacy complains about constants that do not conform to UPPER_CASE naming style:
- `_original_get_adapter` was chosen for consistency with `_original_send`;
- `HTTPStatus_FOUND` was chosen for its proximity to `HTTPStatus.FOUND` (the introduction of this variable is a workaround to the fact that python2.7 does not have `HTTPStatus`.

But I do not have any objection to changing any to upper case.
clslgrnc: Since it seems the semantic of nested mocks is not entirely clear. I'll explain here how I understand it in this PR (and hopefully how it behaves in this PR).

The way I understand it, `Mocker` creates a "fake internet": any call to `requests.Session.send` is intercepted and the chosen fake response corresponding to the input address is returned.
If no response is configured for a given address, it either:
- raises `NoMockAddress`
- return the response from the "real internet" if `_real_http` is set to True (globally for the Mocker or specifically for the given address)

It follows naturally that with nested mocks, the "fake internet" created by a mock is the "real internet" of the mocks it immediately contains.

joshvillbrandt: Woohoo! This fixed my nesting issues. 🎉

Please merge this and release an update to requests-mock on PyPi. Thanks, fam! 👊
jamielennox: heh, sorry i've been moving around a lot and changing jobs. I promise to have a look, just need to be careful that this isn't a behaviour change for anyone. 

Also it's honestly a section of the code that i haven't touched much since i initially wrote it, so i need to wrap my head around it again.
clslgrnc: No worries.
I guess it *will* change the behaviour of nesting since the added test did not pass before the patch, it did not break any other tests though.
I do not know how much this behaviour change will impact the behaviour people relied on (actually I am not even sure what the behaviour was before).

One potential change I haven't checked is that `real_http` now jumps to the mock immediately containing the current mock. Maybe the previous behavior was to completely bypass all mocks (I am not sure).
If this is the case, or a desired feature, we may need two different option:
- one to leave the current mock
- one to leave all mocks
joshvillbrandt: Bump! :)
clslgrnc: Concerning behavior change, since there is no documentation on nesting (correct me if I am wrong), I looked at issues mentioning nesting and found: #27, #44, #82, #105. Let me know if there are other relevant issues.

The first one, #27, seems to show that there was a behavior change between 0.6 and 0.7 (in 0.7 inner mock simply replaces the outter mock) and that at least some people expect mocks to be nestable in particular "I think the ideal behavior would be to try the inner mock first, and if there's no match go to the outer mock" (David Chudzicki).

The behavior I propose is (almost) consistent with both:
- if `real_http` is set to `False` at the mock level, the inner mock replaces the outter mock,
- if `real_http` is set to `True` the inner mock is searched first, then the outter mock is used.

There is a difference though if `real_http` is set to `True` at the resource level in the inner mock then it will search the outter mock instead of jumping directly to an unmocked request (if that is what was happening).

What I propose is also consistent with your suggestion of having a `new` method (without the need of a `new` method):
```python
with self.requests_mock.new() as mocker:
   mocker.get('http://test.com', json={})
```

On #44 I agree with your suggestion to avoid nesting in this case, but this seem to be how some users use requests-mock. I believe this PR would also have solved that user's issue (provided he set `real_http` to `True`).

#82 does not seem relevant and nesting is only mentioned in passing in #105. Still, #105 is relevant because this PR fixes #105:
- if you take just the first commit it forces inner mocks to bypass any outter mock on `real_http`
- if you take the whole PR it pass the current request to the immediate outter mock on `real_http`

I prefer the second one because:
- requests_mock `real_http` itself can be tested without relying on the _real_ internet
- users can define a mock at the test class level that handle all authentication code and block access to the real internet, while test writers can include request mocks without having to know about this outter mock

If you want I can update the documentation to add a section on nesting.
joshvillbrandt: The latest commit continues to work for my mock nesting needs. 👍

Seems like you've done an excellent job reviewing the needs of the community and making sure everything has been addressed. Thanks for your work, @clslgrnc!
clslgrnc: You're welcome, at least it fulfills your (and my) nesting needs (and fixes redirects).
The last force-push was just a rebase and a slight simplification of the PR. 
clslgrnc: With COVID-19 and his new job, @jamielennox probably has other priorities right now (I hope you're ok). If anyone needs nested mocks, as a temporary workaround you can use:
```
pip install git+git://github.com/clslgrnc/requests-mock@6d36219b1cc08111ddbf78ab5df0732874c5812e
```
:warning: **Do not blindly pull code from random people** into your development or continuous integration (or any) environment. If you trust requests-mock, just look at the diff of this PR and if you do not see anything nefarious pull the specific commit you want (not the branch or a tag as it would be fairly easy for me to change the code you use).
I'll refrain from doing any further force push as it may suppress the commit you use.

Then, requests-mock can be reinstalled with:
```
pip install --force-reinstall requests-mock
```

Patch:
diff --git a/requests_mock/mocker.py b/requests_mock/mocker.py
index 0b8565d..9450d12 100644
--- a/requests_mock/mocker.py
+++ b/requests_mock/mocker.py
@@ -89,12 +89,13 @@ class MockerCore(object):
             raise RuntimeError('Mocker has already been started')
 
         self._last_send = requests.Session.send
+        self._last_get_adapter = requests.Session.get_adapter
 
         def _fake_get_adapter(session, url):
             return self._adapter
 
         def _fake_send(session, request, **kwargs):
-            real_get_adapter = requests.Session.get_adapter
+            # mock get_adapter
             requests.Session.get_adapter = _fake_get_adapter
 
             # NOTE(jamielennox): self._last_send vs _original_send. Whilst it
@@ -117,9 +118,13 @@ class MockerCore(object):
                 # requests library rather than the mocking. Let it.
                 pass
             finally:
-                requests.Session.get_adapter = real_get_adapter
+                # restore get_adapter
+                requests.Session.get_adapter = self._last_get_adapter
 
-            return _original_send(session, request, **kwargs)
+            # if we are here it means we must run the real http request
+            # Or, with nested mocks, to the parent mock, that is why we use
+            # _last_send here instead of _original_send
+            return self._last_send(session, request, **kwargs)
 
         requests.Session.send = _fake_send
 


Test patch:
diff --git a/tests/pytest/test_with_pytest.py b/tests/pytest/test_with_pytest.py
index dafccf0..334d73a 100644
--- a/tests/pytest/test_with_pytest.py
+++ b/tests/pytest/test_with_pytest.py
@@ -1,4 +1,12 @@
+try:
+    from http import HTTPStatus
+    HTTP_STATUS_FOUND = HTTPStatus.FOUND
+except ImportError:
+    from httplib import FOUND as HTTP_STATUS_FOUND
+
+import pytest
 import requests
+import requests_mock
 
 
 def test_simple(requests_mock):
@@ -6,6 +14,42 @@ def test_simple(requests_mock):
     assert 'data' == requests.get('https://httpbin.org/get').text
 
 
+def test_redirect_and_nesting():
+    url_inner = "inner_mock://example.test/"
+    url_middle = "middle_mock://example.test/"
+    url_outer = "outer_mock://example.test/"
+    url = "https://www.example.com/"
+    with requests_mock.Mocker() as outer_mock:
+        outer_mock.get(url, text='outer' + url)
+        outer_mock.get(url_outer, text='outer' + url_outer)
+
+        with requests_mock.Mocker(real_http=True) as middle_mock:
+            middle_mock.get(url_middle, text='middle' + url_middle)
+
+            with requests_mock.Mocker() as inner_mock:
+                inner_mock.post(url_inner, status_code=HTTP_STATUS_FOUND, headers={'location': url})
+                inner_mock.get(url, real_http=True)
+
+                assert 'outer' + url == requests.post(url_inner).text  # nosec
+                with pytest.raises(requests_mock.NoMockAddress):
+                    requests.get(url_middle)
+                with pytest.raises(requests_mock.NoMockAddress):
+                    requests.get(url_outer)
+
+            # back to middle mock
+            with pytest.raises(requests_mock.NoMockAddress):
+                requests.post(url_inner)
+            assert 'middle' + url_middle == requests.get(url_middle).text  # nosec
+            assert 'outer' + url_outer == requests.get(url_outer).text  # nosec
+
+        # back to outter mock
+        with pytest.raises(requests_mock.NoMockAddress):
+            requests.post(url_inner)
+        with pytest.raises(requests_mock.NoMockAddress):
+            requests.get(url_middle)
+        assert 'outer' + url_outer == requests.get(url_outer).text  # nosec
+
+
 class TestClass(object):
 
     def configure(self, requests_mock):