Proxy not working

This code will work on my personal laptop, but then when I move to a windows server where we have to use a proxy to access the outside world, I get a 407 error. I added the proxies to this (just like I have setup in my Internet explorer settings that work). Any ideas?

import requests
proxies = {
    "http": "10.51.1.140:8080",
    "https": "10.51.1.140:8080",
}
r = requests.get('https://epfws.usps.gov/ws/resources/epf/version', proxies=proxies)
print(r.text)

Here is what I get:

G:\Python34>python uspsver.py
Traceback (most recent call last):
  File "G:\Python34\lib\site-packages\requests-2.4.3-py3.4.egg\requests\packages
\urllib3\connectionpool.py", line 511, in urlopen
    conn = self._get_conn(timeout=pool_timeout)
  File "G:\Python34\lib\site-packages\requests-2.4.3-py3.4.egg\requests\packages
\urllib3\connectionpool.py", line 231, in _get_conn
    return conn or self._new_conn()
  File "G:\Python34\lib\site-packages\requests-2.4.3-py3.4.egg\requests\packages
\urllib3\connectionpool.py", line 712, in _new_conn
    return self._prepare_conn(conn)
  File "G:\Python34\lib\site-packages\requests-2.4.3-py3.4.egg\requests\packages
\urllib3\connectionpool.py", line 685, in _prepare_conn
    conn.connect()
  File "G:\Python34\lib\site-packages\requests-2.4.3-py3.4.egg\requests\packages
\urllib3\connection.py", line 208, in connect
    self._tunnel()
  File "G:\Python34\lib\http\client.py", line 822, in _tunnel
    message.strip()))
OSError: Tunnel connection failed: 407 Proxy Authentication Required

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "G:\Python34\lib\site-packages\requests-2.4.3-py3.4.egg\requests\adapters
.py", line 364, in send
    timeout=timeout
  File "G:\Python34\lib\site-packages\requests-2.4.3-py3.4.egg\requests\packages
\urllib3\connectionpool.py", line 559, in urlopen
    _pool=self, _stacktrace=stacktrace)
  File "G:\Python34\lib\site-packages\requests-2.4.3-py3.4.egg\requests\packages
\urllib3\util\retry.py", line 265, in increment
    raise MaxRetryError(_pool, url, error)
requests.packages.urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='ep
fws.usps.gov', port=443): Max retries exceeded with url: /ws/resources/epf/versi
on (Caused by ProxyError('Cannot connect to proxy.', OSError('Tunnel connection
failed: 407 Proxy Authentication Required',)))

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "uspsver.py", line 5, in <module>
    r = requests.get('https://epfws.usps.gov/ws/resources/epf/version', proxies=
proxies)
  File "G:\Python34\lib\site-packages\requests-2.4.3-py3.4.egg\requests\api.py",
 line 65, in get
    return request('get', url, *_kwargs)
  File "G:\Python34\lib\site-packages\requests-2.4.3-py3.4.egg\requests\api.py",
 line 49, in request
    response = session.request(method=method, url=url, *_kwargs)
  File "G:\Python34\lib\site-packages\requests-2.4.3-py3.4.egg\requests\sessions
.py", line 459, in request
    resp = self.send(prep, *_send_kwargs)
  File "G:\Python34\lib\site-packages\requests-2.4.3-py3.4.egg\requests\sessions
.py", line 571, in send
    r = adapter.send(request, *_kwargs)
  File "G:\Python34\lib\site-packages\requests-2.4.3-py3.4.egg\requests\adapters
.py", line 415, in send
    raise ConnectionError(e, request=request)
requests.exceptions.ConnectionError: HTTPSConnectionPool(host='epfws.usps.gov',
port=443): Max retries exceeded with url: /ws/resources/epf/version (Caused by P
roxyError('Cannot connect to proxy.', OSError('Tunnel connection failed: 407 Pro
xy Authentication Required',)))

G:\Python34>


The error here is extremely clear: _407 Proxy Authentication Required_.

You need to send some user credentials to the proxy. The real risk here is that the proxy is a Windows domain auth proxy, in which case this will be extremely difficult.


Yep. Clear. But here is the deal. I do NOT have to send in any credentials when using via my internet browser. I will try with user and password and see what it does...I really don't want to have to do that though. 


I'm not surprised. That's because you'll be doing some form of domain auth: maybe NTLM, maybe Kerberos. Either way it is extremely difficult to authenticate with a proxy that requires this kind of auth from Python, and to my knowledge there is no project that does so with requests.


So would trying with user:password actually work? If so what user/password would it need. Using my credentials does not work.


Let's be clear about what you're authenticating to.

If you pass `auth=(user,pass)` that will not work. That attempts to authenticate with the remote server, not the proxy. You'd need to set a Proxy-Authorization header. You can try doing that by setting your proxy URL to `http://user:pass@proxy_ip/`. That _might_ work.


Yep..That is what I just tried. See below. Didn't work if I set it up right. Still get 407 error.

import requests
proxies = {
    "http": "'dmorri':'@XYZ@12345'@proxy.shareddev.acxiom.net:8080",
    "https": "'dmorri':'@XYZ@12345'@proxy.shareddev.acxiom.net:8080",
}
r = requests.get('https://epfws.usps.gov/ws/resources/epf/version', proxies=proxies)
print(r.text)


Uh...why are there quotes in your username and password?


Well my password has @ symbols. :-) So figured I had to put single quotes around it. I can try without. :-)


I think you'll have to try without, but it does raise the question about how well the parsing is going to work here. I wonder what we did last time.


So here is what I get without.

\urllib3\poolmanager.py", line 265, in proxy_from_ur
    return ProxyManager(proxy_url=url, **kw)
  File "G:\Python34\lib\site-packages\requests-2.4.3
\urllib3\poolmanager.py", line 216, in **init**
    'Not supported proxy scheme %s' % proxy.scheme
AssertionError: Not supported proxy scheme dmorri


You left the scheme off the proxy URL. Must begin http://


Same error:

import requests
proxies = {
    "http": "dmorri:@XYZ@12345@http://proxy.shareddev.acxiom.net:8080",
    "https": "dmorri:@XYZ@12345@http://proxy.shareddev.acxiom.net:8080",
}
r = requests.get('https://epfws.usps.gov/ws/resources/epf/version', proxies=proxies)
print(r.text)


Error again is: 

G:\Python34>python uspsver.py
Traceback (most recent call last):
  File "uspsver.py", line 6, in <module>
    r = requests.get('https://epfws.usps.gov/ws/resources/epf/version', proxies=
proxies)
  File "G:\Python34\lib\site-packages\requests-2.4.3-py3.4.egg\requests\api.py",
 line 65, in get
    return request('get', url, *_kwargs)
  File "G:\Python34\lib\site-packages\requests-2.4.3-py3.4.egg\requests\api.py",
 line 49, in request
    response = session.request(method=method, url=url, *_kwargs)
  File "G:\Python34\lib\site-packages\requests-2.4.3-py3.4.egg\requests\sessions
.py", line 459, in request
    resp = self.send(prep, *_send_kwargs)
  File "G:\Python34\lib\site-packages\requests-2.4.3-py3.4.egg\requests\sessions
.py", line 571, in send
    r = adapter.send(request, *_kwargs)
  File "G:\Python34\lib\site-packages\requests-2.4.3-py3.4.egg\requests\adapters
.py", line 331, in send
    conn = self.get_connection(request.url, proxies)
  File "G:\Python34\lib\site-packages\requests-2.4.3-py3.4.egg\requests\adapters
.py", line 239, in get_connection
    proxy_manager = self.proxy_manager_for(proxy)
  File "G:\Python34\lib\site-packages\requests-2.4.3-py3.4.egg\requests\adapters
.py", line 149, in proxy_manager_for
    *_proxy_kwargs)
  File "G:\Python34\lib\site-packages\requests-2.4.3-py3.4.egg\requests\packages
\urllib3\poolmanager.py", line 265, in proxy_from_url
    return ProxyManager(proxy_url=url, *_kw)
  File "G:\Python34\lib\site-packages\requests-2.4.3-py3.4.egg\requests\packages
\urllib3\poolmanager.py", line 216, in **init**
    'Not supported proxy scheme %s' % proxy.scheme
AssertionError: Not supported proxy scheme dmorri

G:\Python34>


Heh, let's be really clear and reprint what I said above:

```
http://user:pass@proxy_ip/
```

The scheme must come before the username and password.


Oops. My bad.


Now I get the 407 error again. 


import requests
proxies = {
    "http": "http://dmorri:@XYZ@12345@proxy.shareddev.acxiom.net:8080",
    "https": "http://dmorri:@XYZ@12345@proxy.shareddev.acxiom.net:8080",
}
r = requests.get('https://epfws.usps.gov/ws/resources/epf/version', proxies=proxies)
print(r.text)


Yup, so that suggests that the procedure doesn't work.

This almost certainly uses NTLM or Kerberos to authenticate. Quick question, does it happen with HTTP as well as HTTPS?


are you prepending an @ to your password?

## 

Kevin Burke
phone: 925.271.7005 | twentymilliseconds.com

On Thu, Nov 13, 2014 at 9:52 AM, dmorri notifications@github.com wrote:

> import requests
> proxies = {
> "http": "http://dmorri:@XYZ https://github.com/XYZ@
> 12345@proxy.shareddev.acxiom.net:8080",
> "https": "http://dmorri:@XYZ https://github.com/XYZ@
> 12345@proxy.shareddev.acxiom.net:8080",
> }
> r = requests.get('https://epfws.usps.gov/ws/resources/epf/version',
> proxies=proxies)
> print(r.text)
> 
> —
> Reply to this email directly or view it on GitHub
> https://github.com/kennethreitz/requests/issues/2338#issuecomment-62935509
> .


Not sure...Do you have a HTTP get call I can try?


Try calling `http://httpbin.org/get`.


Here is what I get on the httpbin.org.

G:\Python34>python uspsver.py
<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN"
"http://www.w3.org/TR/html4/loose.dtd">
<html>
<head>
<meta http-equiv="Content-Type" content="text/html; charset=UTF-8">
<title>Notification: Proxy Authorization Required</title>

<style type="text/css">
<!--
body {
    margin-left: 2%;
    margin-right: 2%;
}
p {
    font-family: verdana, Arial, Helvetica, sans-serif;
    font-size: 12px;
    top: 0px;
    margin-top: 6px;
    margin-bottom: 6px;
}
td {
    font-family: verdana, Arial, Helvetica, sans-serif;
    font-size: 12px;
    top: 0px;
}
.code {
    font-family: courier;
    font-size: 12px;
    top: 0px;
}
h1 {
    font-family: verdana, Arial, Helvetica, sans-serif;
    font-size: 16px;
    margin-top: 24px;
    margin-bottom: 8px;
}
hr {
    margin-top: 2px;
    margin-bottom: 2px;
}
-->
</style>

</head>

<body>
<img src="http://acxpac01.corp.acxiom.net/error/ACXM.gif" border="0">

<h1>This Page Cannot Be Displayed</h1>

<hr>

<p>
Authentication is required to access the Internet using this system.
A valid user ID and password must be entered when prompted.
</p>

<p>If you feel you have reached this page in error and want to have the site rev
iewed, please open a LevelZero request ticket at the <a href="http://levelzero/l
evelzero/">LevelZero Site</a> for the ESOC team.  <b>* Message reported from cwy
ipwsa1 *</b></p>
<p>
If you have questions, need assistance with your login
information, or feel this is an error, please contact
ESOC via a LevelZero ticket
and provide the codes shown below.
</p>

<hr>

<table>
  <tr valign="top">
    <td valign="top" width="10%">Notification&nbsp;codes:&nbsp;</td>
    <td valign="top" class="code" width="90%">(1, PROXY_AUTH_REQUIRED)</td>
  </tr>
</table>

</body>
</html>

G:\Python34>


Right, that's what I expected. Do you know how to use tcpdump or wireshark?


So slightly different, but same type of issue. 

Kevin..What do you mean by pre-appending? The password has two @ symbols. as shown...I just changed to XYZ since I didn't want to post my actual pasword. But the layout is still the same. 
@xxx@11111


I did some dumps with wireshark long ago!!! 


Oh, wait, we don't need it.

Don't print the response, print the response headers for me please.


okay..Hold on.


Sup! You rang?


With the http call I can print it (With https with 407 I cannot). 

G:\Python34>python uspsver.py
{'Content-Length': '1753', 'Proxy-Authenticate': 'NTLM', 'Connection': 'close',
'Content-Type': 'text/html', 'Proxy-Connection': 'close', 'Mime-Version': '1.0',
 'Date': 'Thu, 13 Nov 2014 12:09:08 CST'}


@xxx Oh, that's gotta be the most annoying handle on GitHub. Sorry you got accidentally summoned!