Document how to test Requests without going over the wire

If I am writing unit tests that exercise HTTP requests, it may be useful to mock or stub the external service call, so that the unit tests don't go over the network and introduce latency or fail on a train or a plane. 

It would be useful if Requests provided best practice documentation for:
- how to intercept the request before it goes over the wire, and return a custom response 
- how to determine what Requests was actually ready to send over the wire (potentially by mocking urllib3.connectionpool.HTTP(S)Connection.urlopen?)

Betamax may be appropriate here but it would also be good to explain how to do this without introducing dependencies.


Unless you're on 3.3+ mock is an external dependency so your last sentence is quite wrong. Frankly, if you're performing any parsing of the response and the service is subject to change, e.g. the GitHub API, then mocking is error prone and an awful idea. That aside I still use mocking but not to stub out responses in tests.


Also regarding:

> it may be useful to mock or stub the external service call, so that the unit tests don't go over the network and introduce latency or fail on a train or a plane

You can run HTTPBin locally and tell the tests to use that with an environment variable.

I agree in principle though that by default it shouldn't be talking to the network though. I just thought I'd point out a separate alternative.


I've written this package to test my network libraries: https://github.com/lorien/test_server
It is simple HTTP server based on tornado. It is running in separate python thread. You can configure response (headers, cookies, response body) for any request that your HTTP client sends to the server. Also you have access to details of request that the server received from HTTP client.
You can check out examples of tests here:
- https://github.com/lorien/iob/tree/master/test
- https://github.com/lorien/grab/tree/v06/test


It took me a bit to figure this out too by looking through a lot of different tools while developing my own HTTP mock service framework (https://github.com/BenjamenMeyer/stackInABox) for testing.

So I do agree it could be useful to document how to use the Adapters to intercept stuff this way for testing. They are documented (http://docs.python-requests.org/en/latest/api/#requests.adapters.HTTPAdapter) but having a couple complete examples specific to how tests function would probably be a good thing.

FWIW...:moneybag: 


i'd like this but not for unit tests.

in local dev, we don't want our apiclient (using requests) to visit our "hard to containerise api server"


@airtonix not sure what you mean.

For unit tests you don't want it to hit a real service, so mocking or using the HTTPAdapter to incept (ala requests-mock, responses) is a good thing. You can then control the test and the API Contract being tested against.

For integration/api testing, then yes you want it to go against some kind of services - at one level probably testing against a controlled local environment (e.g docker) but also against live services (staging, production).

In either respect, what you do in the testing - especially unit testing - shouldn't interfere with a developer's ability to run it against live infrastructure (dev, staging, prod environments). They're different use-cases and will be triggered in different manners. Testing needs to be controlled or you lose all value of the test, especially over long term. Dev can be whatever you need.

So if you have a "hard to containerise api server"...

You'll want a tool like StackInABox - or even just using requests-mock, responses, HTTPretty directly - to do your unit testing; but then you'll also have a dev environment running the server for devs to work against.

HTH, and hope I didn't ramble too much.

@BenjamenMeyer Nah sometimes things don't have to be dogmatic. It's actually really stiffling and annoying.

We're doing this instead now... since it's obviously too hard to mess with requests;

in prod:
```
frontend view > mobxStore.getFoos > networkClient.getFoos `GET "/foos"` > DjangoApiView `/foos` > Django.ServerToServerClient.getFoos > requests `GET https://remoteserver/foos`
```

in staging/dev:
```
frontend view > mobxStore.getFoos > networkClient.getFoos `GET "/foos"` > DjangoApiView `/foos` > Django.ServerToServerClient.getFoos > FakeResponse FactoryBoy.generateFoo
```

We do this by having a module name in `settings.dev` like : `settings.API_CLIENT_STRATEGY='apiclient.mocks' ` which is a bunch of actions that as you see above returns http like responses with factoryboy or faker.

In `settings.prod` its `settings.API_CLIENT_STRATEGY='apiclient.http'` which is a bunch of actions that use `requests` and returns real responses from our 'hard-to-containerise' service.

And in our `app.apiclient`  we use importlib to 

But hey, thanks for trying but you missed the part where I **do not want to do this in a unit test**.  

also not every reply is about you.

We have well supported framework for mocking up responses:
https://github.com/getsentry/responses

@nateprewitt 
I believe this issue could be closed
or if you think this information should be needed in the docs, then I would be glad to submit a PR

I think there are multiple options at this point. The original task here was to look at documenting how to do this through the adapters interface. Given there's been no progress on this in ~8 years though, we'll resolve this in favor of the alternatives. Thanks everyone!