kennethreitz/requests — pull request #461 by umbrae (2012-02-28 17:15:36+00:00)
Title: Decode URL to utf-8 before joining.

To avoid UnicodeDecodeError's like on http://blip.fm/~1abvfu

Note: This doesn't have a test, which sucks, because for the life of me I can't seem to figure out how to trigger this error with httpbin. Maybe flask has better handling of UTF-8 characters than the server used at blip.fm and others? Really not sure.

Here's an ad-hoc test for you:

Before:

>>> import requests
>>> requests.head('http://blip.fm/~1abvfu', allow_redirects=True)
Traceback (most recent call last):
File "", line 1, in
File "requests/api.py", line 73, in head
return request('head', url, **kwargs)
File "requests/api.py", line 39, in request
return s.request(method=method, url=url, **kwargs)
File "requests/sessions.py", line 200, in request
r.send(prefetch=prefetch)
File "requests/models.py", line 559, in send
self._build_response(r)
File "requests/models.py", line 232, in _build_response
url = urljoin(r.url, url)
File "/System/Library/Frameworks/Python.framework/Versions/2.6/lib/python2.6/urlparse.py", line 219, in urljoin
params, query, fragment))
File "/System/Library/Frameworks/Python.framework/Versions/2.6/lib/python2.6/urlparse.py", line 184, in urlunparse
return urlunsplit((scheme, netloc, url, query, fragment))
File "/System/Library/Frameworks/Python.framework/Versions/2.6/lib/python2.6/urlparse.py", line 190, in urlunsplit
url = '//' + (netloc or '') + url
UnicodeDecodeError: 'ascii' codec can't decode byte 0xe2 in position 62: ordinal not in range(128)

After:

>>> import requests
>>> requests.head('http://blip.fm/~1abvfu', allow_redirects=True)