Repository: urllib3/urllib3

Issue:
Proxy Headers don't get applied when using ProxyManager.connection_from_url()
Brought up in the Requests IRC room: it turns out Requests no longer correctly applies Proxy-Authentication headers to HTTP messages via proxies.

If you use `ProxyManager.urlopen()`, [this block of code](https://github.com/shazow/urllib3/blob/master/urllib3/poolmanager.py#L247-L253) applies some HTTP headers:

``` python
if u.scheme == "http":
    # It's too late to set proxy headers on per-request basis for
    # tunnelled HTTPS connections, should use
    # constructor's proxy_headers instead.
    kw['headers'] = self._set_proxy_headers(url, kw.get('headers',
                                                        self.headers))
    kw['headers'].update(self.proxy_headers)
```

But no such code back exists when using `ProxyManager.connection_from_url`.

I'm open to providing code to fix this, but am not sure I can do so tonight, so if someone else would like to they should feel free.



Discussion:
Sadness. Fix appreciated whenever you can provide. :)


Resolving patch:
diff --git a/urllib3/connectionpool.py b/urllib3/connectionpool.py
--- a/urllib3/connectionpool.py
+++ b/urllib3/connectionpool.py
@@ -461,6 +461,13 @@ def urlopen(self, method, url, body=None, headers=None, retries=3,
 
         conn = None
 
+        # Merge the proxy headers. Only do this in HTTP. We have to copy the
+        # headers dict so we can safely change it without those changes being
+        # reflected in anyone else's copy.
+        if self.scheme == 'http':
+            headers = headers.copy()
+            headers.update(self.proxy_headers)
+
         try:
             # Request a connection from the queue
             conn = self._get_conn(timeout=pool_timeout)
diff --git a/urllib3/poolmanager.py b/urllib3/poolmanager.py
--- a/urllib3/poolmanager.py
+++ b/urllib3/poolmanager.py
@@ -245,12 +245,11 @@ def urlopen(self, method, url, redirect=True, **kw):
         u = parse_url(url)
 
         if u.scheme == "http":
-            # It's too late to set proxy headers on per-request basis for
-            # tunnelled HTTPS connections, should use
-            # constructor's proxy_headers instead.
+            # For proxied HTTPS requests, httplib sets the necessary headers
+            # on the CONNECT to the proxy. For HTTP, we'll definitely
+            # need to set 'Host' at the very least.
             kw['headers'] = self._set_proxy_headers(url, kw.get('headers',
                                                                 self.headers))
-            kw['headers'].update(self.proxy_headers)
 
         return super(ProxyManager, self).urlopen(method, url, redirect, **kw)
 


Test patch:
diff --git a/test/with_dummyserver/test_socketlevel.py b/test/with_dummyserver/test_socketlevel.py
--- a/test/with_dummyserver/test_socketlevel.py
+++ b/test/with_dummyserver/test_socketlevel.py
@@ -205,11 +205,42 @@ def echo_socket_handler(listener):
                              b'GET http://google.com/ HTTP/1.1',
                              b'Host: google.com',
                              b'Accept-Encoding: identity',
-                             b'Accept: */*',
                              b'',
                              b'',
                          ]))
 
+    def test_headers(self):
+        def echo_socket_handler(listener):
+            sock = listener.accept()[0]
+
+            buf = b''
+            while not buf.endswith(b'\r\n\r\n'):
+                buf += sock.recv(65536)
+
+            sock.send(('HTTP/1.1 200 OK\r\n'
+                      'Content-Type: text/plain\r\n'
+                      'Content-Length: %d\r\n'
+                      '\r\n'
+                      '%s' % (len(buf), buf.decode('utf-8'))).encode('utf-8'))
+            sock.close()
+
+        self._start_server(echo_socket_handler)
+        base_url = 'http://%s:%d' % (self.host, self.port)
+
+        # Define some proxy headers.
+        proxy_headers = {'For The Proxy': 'YEAH!'}
+        proxy = proxy_from_url(base_url, proxy_headers=proxy_headers)
+
+        conn = proxy.connection_from_url('http://www.google.com/')
+
+        r = conn.urlopen('GET', 'http://www.google.com/', assert_same_host=False)
+
+        self.assertEqual(r.status, 200)
+        # FIXME: The order of the headers is not predictable right now. We
+        # should fix that someday (maybe when we migrate to
+        # OrderedDict/MultiDict).
+        self.assertTrue(b'For The Proxy: YEAH!\r\n' in r.data)
+
 
 class TestSSL(SocketDummyServerTestCase):