CERTIFICATE_VERIFY_FAILED with 2.5.2, works with 2.5.1

I'm having a somewhat odd issue. A get() request seems to work fine with requests-2.5.1, but after upgrading to requests 2.5.2, the same URL leads to `CERTIFICATE_VERIFY_FAILED`. cc @mpharrigan

```
$ pip install requests==2.5.1
[ ... snip, installs just fine ...]

$ python -c 'import requests; print(requests.get("http://conda.binstar.org/omnia/linux-64/fftw3f-3.3.3-1.tar.bz2"))'
<Response [200]>

$ pip install requests==2.5.2
[ ... snip, installs just fine ...]

$ python -c 'import requests; print(requests.get("http://conda.binstar.org/omnia/linux-64/fftw3f-3.3.3-1.tar.bz2"))'
Traceback (most recent call last):
  File "/Users/rmcgibbo/miniconda/envs/3.4.2/lib/python3.4/site-packages/requests/packages/urllib3/connectionpool.py", line 544, in urlopen
    body=body, headers=headers)
  File "/Users/rmcgibbo/miniconda/envs/3.4.2/lib/python3.4/site-packages/requests/packages/urllib3/connectionpool.py", line 341, in _make_request
    self._validate_conn(conn)
  File "/Users/rmcgibbo/miniconda/envs/3.4.2/lib/python3.4/site-packages/requests/packages/urllib3/connectionpool.py", line 762, in _validate_conn
    conn.connect()
  File "/Users/rmcgibbo/miniconda/envs/3.4.2/lib/python3.4/site-packages/requests/packages/urllib3/connection.py", line 238, in connect
    ssl_version=resolved_ssl_version)
  File "/Users/rmcgibbo/miniconda/envs/3.4.2/lib/python3.4/site-packages/requests/packages/urllib3/util/ssl_.py", line 256, in ssl_wrap_socket
    return context.wrap_socket(sock, server_hostname=server_hostname)
  File "/Users/rmcgibbo/miniconda/envs/3.4.2/lib/python3.4/ssl.py", line 364, in wrap_socket
    _context=self)
  File "/Users/rmcgibbo/miniconda/envs/3.4.2/lib/python3.4/ssl.py", line 578, in __init__
    self.do_handshake()
  File "/Users/rmcgibbo/miniconda/envs/3.4.2/lib/python3.4/ssl.py", line 805, in do_handshake
    self._sslobj.do_handshake()
ssl.SSLError: [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed (_ssl.c:600)

During handling of the above exception, another exception occurred:
[... snip ...]
```


FWIW, this is with python 3.4.2 and OS X 10.10.


@t-8ch seems related to the urllib3 patch to disable built-in hostname verification possibly.


Would it be helpful for me to run a git-bisect or something?


Not really. The change is upstream of us. Thanks for the bug though.


@sigmavirus24 I would rather blame the new ciphersuite or cert bundle (not tested)


Could easily be the new cert bundle.


I'm assuming that you hand-typed that code @rmcgibbo? The reason I ask is because you're accessing HTTP urls there.


I cannot reproduce this on my copy of 2.5.2.


That suggests it's not the cert bundle.


@Lukasa I can now reproduce it, the URLs are right, there seems to be an redirect


So there is. I still can't reproduce though.


@Lukasa Are you running from a git checkout? It works for me there to, but not from a wheel, will try a sdist


Nope, from a wheel.

``` python
Python 2.7.9 (default, Dec 19 2014, 06:00:59) 
[GCC 4.2.1 Compatible Apple LLVM 6.0 (clang-600.0.56)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> import requests
>>> requests.__version__
'2.5.2'
>>> requests.__file__
'/usr/local/lib/python2.7/site-packages/requests/__init__.pyc'
>>> r = requests.get("http://conda.binstar.org/omnia/linux-64/fftw3f-3.3.3-1.tar.bz2")
>>> r.status_code
200
>>> r.url
u'https://binstar-cio-packages-prod.s3.amazonaws.com/531fd36fe1dad154c8ea89c5/54b0d1045e76836c22735094?Signature=JRp8nPqRc6wXijqGg5JjDnIytz8%3D&Expires=1424763499&AWSAccessKeyId=ASIAIIYL6XQN5BMSPEFA&response-content-disposition=attachment%3B%20filename%3Dfftw3f-3.3.3-1.tar.bz2&response-content-type=application/x-tar&x-amz-security-token=AQoDYXdzEM///////////wEa4APi0P1wChyd8cFWFddkWZVgQYQQPAli9WBPWotWMA77n%2BqFSO92c2mb607CMadC7Jdo21sJVmtuKu4gM2vQhzX8Uy8NXWHtrBpkAgulTshstH1oFEcrkfV2wVtcxiHPrWTrEw7UTHAqDBQppLNHIHTQjs%2Bks0d/yQ//i6iCJTulVjNUWsXv%2BGre%2BMX2s%2BetTldscc2vbHsrzLG%2BvlOGrKbgUaDzLHWa7RgA6z4DxJdu0inmSPGIoV02fFonVRrwjI3qUrw50dmIFIW5SFxN9XYnDnXf1DiJ2b3vKnrj9I%2BYEY%2ByP5pHpSWvF%2BIRrmBDX6KHf4QKF5%2Bo01Y2Oeqq%2BhQncFFdEt6QBU6Eqa%2B3kb6iwdJdPAXr7HOgd5TugixLDNnKaWuF%2B1grizrD8Drz6nYhb0sgZjazDmjNSFpNXwCyYDWQNNIL7o13Sh6q2fB8sWD5oCEJu22HB4CWBxq3AC/APi86cPSZ/5RiRnv2pz%2BBB7HGNwTZveJaSfSX9ZR2pq6oZOT5kpP2aJrj4wwvE7BC0kf%2B2au4B%2BGH5uHNZvm42%2BwWophhx9NcIjdX/F9g%2B8cnkrx9nooNcB%2BVzxgXZG%2BCzPuhdFbcRFzWaqz0W/T5eJWVpmCIcKpGlIibqcwAjGAg16%2BwpwU%3D'
```


Potentially also relevant:

``` python
>>> import ssl
>>> ssl.OPENSSL_VERSION
'OpenSSL 1.0.2 22 Jan 2015'
```


It seems to be the bundle.


@t-8ch What makes you say that?


It works with the old one and does not with the new one, also it works with my system bundle.
(requests and curl)


Aha, yep, can now reproduce. Don't know what I was doing differently before. =(


So the cert chain is:

```
 0 s:/C=US/ST=Washington/L=Seattle/O=Amazon.com Inc./CN=*.s3.amazonaws.com
   i:/C=US/O=VeriSign, Inc./OU=VeriSign Trust Network/OU=Terms of use at https://www.verisign.com/rpa (c)10/CN=VeriSign Class 3 Secure Server CA - G3
 1 s:/C=US/O=VeriSign, Inc./OU=VeriSign Trust Network/OU=Terms of use at https://www.verisign.com/rpa (c)10/CN=VeriSign Class 3 Secure Server CA - G3
   i:/C=US/O=VeriSign, Inc./OU=VeriSign Trust Network/OU=(c) 2006 VeriSign, Inc. - For authorized use only/CN=VeriSign Class 3 Public Primary Certification Authority - G5
 2 s:/C=US/O=VeriSign, Inc./OU=VeriSign Trust Network/OU=(c) 2006 VeriSign, Inc. - For authorized use only/CN=VeriSign Class 3 Public Primary Certification Authority - G5
   i:/C=US/O=VeriSign, Inc./OU=Class 3 Public Primary Certification Authority
```


That Class 3 Public Primary Certification Authority - G5 certificate is in the bundle. The one Mozilla uses as its trust root when I access the page in Firefox is definitely 100% in that bundle. Serial numbers match and everything.


Weirdly though, if I pass that certificate to curl to use to validate the connection, it doesn't like it.


A warning that so far I've not been able to reproduce this _in requests_ on either Windows or OS X, presumably because their built in cert-stores are saving the day.


Also, running `openssl s_client` seems to have no problems with that bundle:

```
ubuntu@cb2ubuntu:~/garpd$ cat /usr/local/lib/python2.7/dist-packages/requests/__init__.py | grep version
__version__ = '2.5.2'
```

```
ubuntu@cb2ubuntu:~/garpd$ openssl s_client -CApath /usr/local/lib/python2.7/dist-packages/requests/cacert.pem -connect binstar-cio-packages-prod.s3.amazonaws.com:443                                                                                                                [8/8]
CONNECTED(00000003)
depth=3 C = US, O = "VeriSign, Inc.", OU = Class 3 Public Primary Certification Authority
verify return:1
depth=2 C = US, O = "VeriSign, Inc.", OU = VeriSign Trust Network, OU = "(c) 2006 VeriSign, Inc. - For authorized use only", CN = VeriSign Class 3 Public Primary Certification Authority - G5
verify return:1
depth=1 C = US, O = "VeriSign, Inc.", OU = VeriSign Trust Network, OU = Terms of use at https://www.verisign.com/rpa (c)10, CN = VeriSign Class 3 Secure Server CA - G3
verify return:1
depth=0 C = US, ST = Washington, L = Seattle, O = Amazon.com Inc., CN = *.s3.amazonaws.com
verify return:1
---
Certificate chain
 0 s:/C=US/ST=Washington/L=Seattle/O=Amazon.com Inc./CN=*.s3.amazonaws.com
   i:/C=US/O=VeriSign, Inc./OU=VeriSign Trust Network/OU=Terms of use at https://www.verisign.com/rpa (c)10/CN=VeriSign Class 3 Secure Server CA - G3
 1 s:/C=US/O=VeriSign, Inc./OU=VeriSign Trust Network/OU=Terms of use at https://www.verisign.com/rpa (c)10/CN=VeriSign Class 3 Secure Server CA - G3
   i:/C=US/O=VeriSign, Inc./OU=VeriSign Trust Network/OU=(c) 2006 VeriSign, Inc. - For authorized use only/CN=VeriSign Class 3 Public Primary Certification Authority - G5
 2 s:/C=US/O=VeriSign, Inc./OU=VeriSign Trust Network/OU=(c) 2006 VeriSign, Inc. - For authorized use only/CN=VeriSign Class 3 Public Primary Certification Authority - G5
   i:/C=US/O=VeriSign, Inc./OU=Class 3 Public Primary Certification Authority
---
Server certificate
-----BEGIN CERTIFICATE-----
MIIFQTCCBCmgAwIBAgIQGHBX7tZDXzmvfSkeROrx7DANBgkqhkiG9w0BAQUFADCB
tTELMAkGA1UEBhMCVVMxFzAVBgNVBAoTDlZlcmlTaWduLCBJbmMuMR8wHQYDVQQL
ExZWZXJpU2lnbiBUcnVzdCBOZXR3b3JrMTswOQYDVQQLEzJUZXJtcyBvZiB1c2Ug
YXQgaHR0cHM6Ly93d3cudmVyaXNpZ24uY29tL3JwYSAoYykxMDEvMC0GA1UEAxMm
VmVyaVNpZ24gQ2xhc3MgMyBTZWN1cmUgU2VydmVyIENBIC0gRzMwHhcNMTQwNDA5
MDAwMDAwWhcNMTUwNDA5MjM1OTU5WjBrMQswCQYDVQQGEwJVUzETMBEGA1UECBMK
V2FzaGluZ3RvbjEQMA4GA1UEBxQHU2VhdHRsZTEYMBYGA1UEChQPQW1hem9uLmNv
bSBJbmMuMRswGQYDVQQDFBIqLnMzLmFtYXpvbmF3cy5jb20wggEiMA0GCSqGSIb3
DQEBAQUAA4IBDwAwggEKAoIBAQCyIdaCeebmUg7oowAEkJOGAkE9KA7f/Kpsbexn
sD0v/W2Hbq7Kmys8LD9bs6RX4YNIr/Cx0i4gQlymmVXy/OhgrvSpl/lbmHzFXF30
UF2/L6NWkbkca2QbmolYBjYHngblx/gRQw6XGSui2Ql8q6W5IOz1EyHUZOhcr5W8
x76JtY4r5/uav+2WO9pgtGEL4aROQfE7R/399OvkUCabcTvaG9N0TMBLTdB/mWyD
GlnHSwWl67lH1HPr429iz/2cPP7l3eq1V1PNq25w5JCV2kySmq5d0XKt4cy5mMh/
Og2vcwyj31u8B4fzyGWxQAXLs10wWF9xdVNHrJwoBD9jeiWDAgMBAAGjggGUMIIB
kDAJBgNVHRMEAjAAMEMGA1UdIAQ8MDowOAYKYIZIAYb4RQEHNjAqMCgGCCsGAQUF
BwIBFhxodHRwczovL3d3dy52ZXJpc2lnbi5jb20vY3BzMEUGA1UdHwQ+MDwwOqA4
oDaGNGh0dHA6Ly9TVlJTZWN1cmUtRzMtY3JsLnZlcmlzaWduLmNvbS9TVlJTZWN1
cmVHMy5jcmwwHQYDVR0lBBYwFAYIKwYBBQUHAwEGCCsGAQUFBwMCMB8GA1UdIwQY
MBaAFA1EXBZTRMGCfh0gqyX0AWPYvnmlMHYGCCsGAQUFBwEBBGowaDAkBggrBgEF
BQcwAYYYaHR0cDovL29jc3AudmVyaXNpZ24uY29tMEAGCCsGAQUFBzAChjRodHRw
Oi8vU1ZSU2VjdXJlLUczLWFpYS52ZXJpc2lnbi5jb20vU1ZSU2VjdXJlRzMuY2Vy
MA4GA1UdDwEB/wQEAwIFoDAvBgNVHREEKDAmghIqLnMzLmFtYXpvbmF3cy5jb22C
EHMzLmFtYXpvbmF3cy5jb20wDQYJKoZIhvcNAQEFBQADggEBAD2yDlI/JHDW9LNT
rsvy1lnS8H0IT8Zc+z9Imd5zEEqBs2G1beCtM9U4o/MDEao95DWfRck3Gx428fPv
bsabSwJHtSpGLQiWi/UwnxN0p5Lz6tQVaglBqlsvm4ZGHdS94hSaYwd4nUZ+Wpo8
hhCk44lVjwD0hTqr4G08XQiS/mlOY2422zo6+ULw+YG6ocMtVTe+VsL3V7dLRYgN
wV15Z5GLL4f50hbUHQAjdFHMtDkIQTWu0l7SJB6ueQBxoBNJoHC89IZMom0Oy9WL
1UNYgBTsad76ql/K3feTPJodalB1RXbEwSgc4pAC1/rtlfoZewZvNqANMxYc7k7G
ufhUTyk=
-----END CERTIFICATE-----
subject=/C=US/ST=Washington/L=Seattle/O=Amazon.com Inc./CN=*.s3.amazonaws.com
issuer=/C=US/O=VeriSign, Inc./OU=VeriSign Trust Network/OU=Terms of use at https://www.verisign.com/rpa (c)10/CN=VeriSign Class 3 Secure Server CA - G3
---
No client certificate CA names sent
---
SSL handshake has read 4632 bytes and written 455 bytes
---
New, TLSv1/SSLv3, Cipher is ECDHE-RSA-AES128-SHA
Server public key is 2048 bit
Secure Renegotiation IS supported
Compression: NONE
Expansion: NONE
SSL-Session:
    Protocol  : TLSv1.2
    Cipher    : ECDHE-RSA-AES128-SHA
    Session-ID: 65EDD443247002AE9E1DCBE70147AA14DF818C026DD0DC3A07575DEEEFA01895
    Session-ID-ctx:
    Master-Key: 4814563FB3450B10F25DB657DC158FFF4CE9B1601057F3CB552BC5B881D40FD2FA97F8B8A0BCB86B6E1D830C3D647299
    Key-Arg   : None
    PSK identity: None
    PSK identity hint: None
    SRP username: None
    Start Time: 1424769860
    Timeout   : 300 (sec)
    Verify return code: 0 (ok)
---
```


But cert verification definitely fails with requests. So that's perplexing.


At least we get the same leaf cert with s_client and requests.


@t-8ch To verify: does your bundle succeed with `s_client`?


Now I am _really_ confused:


Installed from wheel:

py2 + 2.5.1: works
py3.3 + 2.5.1: works
py3.4 + 2.5.1: works

py2 + 2.5.2: breaks
py3.3 + 2.5.2: breaks
py3.4 + 2.5.2: breaks

Git checkout:

py2 + 2.5.1: works
py3 + 2.5.1: works

py2 + 2.5.2: breaks
py3.3 + 2.5.2: breaks
py3.4 + 2.5.2: works


All on linux with 3.4.2 and 2.7.9


Well that's extremely perplexing. Can you diff the cacert.pem files in each case?