Implement per-host proxies

This change allows the proxy dict to be have entries of the form
`{'<scheme>://<hostname>': '<proxy>'}`.  Host-specific proxies will be used in
preference to the scheme-specific proxies (i.e., proxy dict entries with keys
like 'http' or 'https').

Fixes #2722


This looks reasonable enough to me.

Thoughts @sigmavirus24?


Nope. This thoroughly breaks backwards compatibility. This changes the behaviour to **only** behave for per-host which means anyone passing `proxies={'http': ..., 'https': ...}` will see everything fail as soon as they upgrade.

We should support both, not one or the other.


Closing/re-opening to trigger a new CI build


Also there aren't any tests for this


This does support both.

It does a lookup for `u.scheme+'://'+u.hostname`, and if that does not exist falls back to the get for `u.scheme`. Or have I missed something?


@Lukasa - that's what I intended, and what I'm sure the code does.  Fall back to the scheme lookup if the scheme+hostname lookup fails.  Would you rather I make it 5 lines of code to make that clearer?


@sigmavirus24 - this is my first contribution to requests.  How does testing work in this package?


I added a note to the documentation about this.


> It does a lookup for u.scheme+'://'+u.hostname, and if that does not exist falls back to the get for u.scheme. Or have I missed something?

Nesting it like that made me miss it without having had enough coffee. It should be done separately to be very explicit.

As for tests, they go in the `test_requests.py` file.


@sigmavirus24, I didn't see any tests in test_requests.py for the current proxies argument to request methods.  I saw a few places where it is set from the urllib getproxies function, presumably to do some actual network calls.  I also saw two tests for the no_proxy environment variable.  Is there a test for the proxies argument to a request that I missed?  I'm happy to write one, just wanted to make sure I didn't miss it (and I was looking for a model for how to write a test that is consistent with your framework).


Testing proxies code is tricky, normally, because of our tendency to do 'live' testing, and because this function is very side-effect-y. The best way to do it might be to pull this 'find the correct proxy url' logic into its own function in `util.py` and then unit test that specifically to verify the logic. I think that would keep everyone happy. =)


@Lukasa, I've done that now.  I'm not very clear on how this plays with the environment proxies, especially in rebuild_proxies() in the sessions code (https://github.com/kennethreitz/requests/blob/master/requests/sessions.py#L228).  Any help from someone that already knows the call flow and how the proxies are changed at each step is appreciated.  I think I don't need to do anything, but I'm not absolutely sure.


One note here, otherwise fine.

The change you've made should be ok, but we do need to verify it. I can't do that right now, but I'll hopefully get to it Monday.


@Lukasa - a friendly ping about reviewing this.  Thanks again for helping this along!


One small docs note. Otherwise, a live test appears to validate that this works. =)


I changed the docs to emphasize that the per-host proxy matches only on the scheme and exact hostname.  Thanks!


Ok, I'm officially happy with this. @sigmavirus24, can I get you to take a quick look?


Looks good to me. Thanks for your hard work and diligence @jasongrout 


Hurrah! @sigmavirus24 do we want to merge this before 2.8.0 or hold off until the release after?


May as well pull it into 2.8.0. Seems like @jasongrout needs this sooner rather than later.


Cool!


Thanks!


@Lukasa - just curious, is there a planned release date for 2.8.0?


There is not. I'm hoping to get to it this week, maybe tomorrow, but currently 2.8.0's release is a best-effort kind of thing.