getsentry/sentry — pull request #1307 by vperron (2014-11-05 17:24:30+00:00)
Title: Replace urllib2 with python-requests

Hello,

I came across a very naughty bug when trying to access to an SNI (Server Name Indication) HTTPS website, which is the case for every Amazon Cloudfront-enabled website as a default, for instance, and will increasingly happen in the future.

Here is a dump of what happens on the Sentry server:

```python
[05/Nov/2014 10:31:46] "GET /api/xxx/yyy/poll/?cursor=1415030294 HTTP/1.0" 200 1461
Fetching remote source u'https://dashboard.locarise.com/index.html'
Disabling sources to dashboard.locarise.com for 300s
Traceback (most recent call last):
File "/home/sentry/env/local/lib/python2.7/site-packages/sentry/tasks/fetch_source.py", line 167, in fetch_url
timeout=settings.SENTRY_SOURCE_FETCH_TIMEOUT)
File "/home/sentry/env/local/lib/python2.7/site-packages/sentry/http.py", line 83, in safe_urlopen
return opener.open(req, timeout=timeout)
File "/usr/lib/python2.7/urllib2.py", line 401, in open
response = self._open(req, data)
File "/usr/lib/python2.7/urllib2.py", line 419, in _open
'_open', req)
File "/usr/lib/python2.7/urllib2.py", line 379, in _call_chain
result = func(*args)
File "/usr/lib/python2.7/urllib2.py", line 1219, in https_open
return self.do_open(httplib.HTTPSConnection, req)
File "/usr/lib/python2.7/urllib2.py", line 1181, in do_open
raise URLError(err)
URLError:
```

Well, the error is easy to reproduce: any urllib2.urlopen will fail when trying to do this.

And that's entirely normal, urllib2 does not support SNI in python 2.7.8.
It's planned to be backported for the next release (2.7.9), CF http://legacy.python.org/dev/peps/pep-0466/

Still, using requests, pyopenssl, ndg-httpsclient and pyasn1, we can support it, and that's what's explained in this PEP466.

Here is the SO thread that explains how to fix the issue:
https://stackoverflow.com/questions/18578439/using-requests-with-tls-doesnt-give-sni-support/18579484#18579484

This simple patch replaces urllib2 with python-requests, which also simplifies the code. The tests are all passing.

Now, do we wait for Python 2.7.9 instead, that's the question.