shazow/urllib3 — Issue #38: URL parsing errors
Author: tokarina
Created: 2012-01-23 13:18:41+00:00
State: open

This is an example of URL that urllib3 fails to parse correctly. It appears that it un-encodes the URL and then gets confused about the colon symbol in the mailto: statement that is part of the path.

Strangely enough, requests generates a different error for the same URL. For urllib3, it makes the pool think that the URL is from a different host. For requests, it thinks the other host is a new port number... and tries to int(string)...

Is this a known issue?

>>> http_pool = urllib3.connection_from_url('http://stats.e-go.gr/rx.asp?nWebSrvID=100230&nCatID=23425&nLevelId=-20&target=mailto%3Aadvertising%40pegasusinteractive%2Egr')
>>> r = http_pool.get_url('http://stats.e-go.gr/rx.asp?nWebSrvID=100230&nCatID=23425&nLevelId=-20&target=mailto%3Aadvertising%40pegasusinteractive%2Egr')Traceback (most recent call last):
File "", line 1, in
File "urllib3/request.py", line 136, in get_url
**urlopen_kw)
File "urllib3/request.py", line 78, in request_encode_url
return self.urlopen(method, url, **urlopen_kw)
File "urllib3/connectionpool.py", line 410, in urlopen
retries - 1, redirect, assert_same_host)
File "urllib3/connectionpool.py", line 341, in urlopen
raise HostChangedError(host, url, retries - 1)
urllib3.exceptions.HostChangedError: Connection pool with host 'http://stats.e-go.gr' tried to open a foreign host: mailto:advertising@pegasusinteractive.gr
>>>

>>> import requests
>>> r = requests.get('http://stats.e-go.gr/rx.asp?nWebSrvID=100230&nCatID=23425&nLevelId=-20&target=mailto%3Aadvertising%40pegasusinteractive%2Egr')Traceback (most recent call last):
File "", line 1, in
File "/usr/local/lib/python2.7/dist-packages/requests/api.py", line 50, in get
return request('get', url, **kwargs)
File "/usr/local/lib/python2.7/dist-packages/requests/api.py", line 38, in request
return s.request(method=method, url=url, **kwargs)
File "/usr/local/lib/python2.7/dist-packages/requests/sessions.py", line 200, in request
r.send(prefetch=prefetch)
File "/usr/local/lib/python2.7/dist-packages/requests/models.py", line 514, in send
self._build_response(r)
File "/usr/local/lib/python2.7/dist-packages/requests/models.py", line 253, in _build_response
request.send()
File "/usr/local/lib/python2.7/dist-packages/requests/models.py", line 430, in send
conn = self._poolmanager.connection_from_url(url)
File "/usr/local/lib/python2.7/dist-packages/requests/packages/urllib3/poolmanager.py", line 94, in connection_from_url
scheme, host, port = get_host(url)
File "/usr/local/lib/python2.7/dist-packages/requests/packages/urllib3/connectionpool.py", line 524, in get_host
port = int(port)
ValueError: invalid literal for int() with base 10: 'advertising%40pegasusinteractive.gr'
>>>