Review new httpbin package and pytest-httpbin for testing

https://github.com/kevin1024/pytest-httpbin


That's still rather slow. [Betamax](/sigmavirus24/betamax) can cover our needs and still be faster than using that. If you're concerned about the speed of our tests, I'll be happy to add betamax to the mix.


This is still making actual HTTP requests, though. How much can we trust a Betamax-style test suite? I feel like I'd ALWAYS want to run it against an HTTPbin just to be sure anyway. 


HTTPbin changes exactly how often? That's exactly how often you would have to run the tests against the site or a local copy. You can also control with what precision betamax will match a request in a recorded cassette so you can send multiple posts with varying bodies and be sure you'll get the right response.

As for trust, @dgouldin might be able to tell us how much he trusts them. I use them, and the equivalent tool in Ruby (VCR) is trusted quite heavily by quite a large number of people including several corporations I can think of off the top of my head.


I wrote pytest-httpbin to test [VCR.py](https://github.com/kevin1024/vcrpy), which is similar to Betamax, because I wanted to make sure I was mocking httplib correctly.

I notice that Betamax replaces the requests HTTP adapter with a BetamaxAdapter, so you wouldn't be able to test the built-in requests.adapters.HTTPAdapter with Betamax tests.

Incidentally, VCR.py replaces the HTTPConnection from httplib, monkeypatching at a higher level, so it might actually be a better choice for that particular case.


## It replaces it only to intercept PreparedRequests. It still continues to use the HTTPAdapter (or any adapter registered) to make the real requests. You're just replacing something at a lower level and making assumptions about every user's use case. There are users of the FTP Adapter for whom vcr.py is not useful while Betamax would work.

Sent from my Android device with K-9 Mail. Please excuse my brevity.


I see.  I didn't know requests supported FTP, that's very cool.

You're obviously welcome to use whatever tools you think are best.  I didn't mean to start an argument here!

BTW - perhaps Betamax itself would benefit from pytest-httpbin.

> ## It replaces it only to intercept PreparedRequests. It still continues to use the HTTPAdapter (or any adapter registered) to make the real requests. You're just replacing something at a lower level and making assumptions about every user's use case. There are users of the FTP Adapter for whom vcr.py is not useful while Betamax would work.
> 
> Sent from my Android device with K-9 Mail. Please excuse my brevity.
> —
> Reply to this email directly or view it on GitHub.


I'm sincerely sorry if my response came across as argumentative @kevin1024. That really wasn't my intention. As for whether or not Betamax should use pytest-httpbin we should probably discuss that on an issue in Betamax. :)


No harm done, it's hard to read tone over the internet :)

> It still continues to use the HTTPAdapter (or any adapter registered) to make the real requests

The real requests are only going to happen if the cassette doesn't contain the request, right?  So if the request hits the cassette, then the code inside the adapter isn't going to be tested.  Of course if you delete the cassette and re-record, then the test will hit all the code.


Yes. Also you can have a re-record interval so that you don't have to concern yourself with deleting the cassettes.


This is a duplicate of #2184. Closing to centralize discussion in one place.