Session requests should respect verify=False

Using a website that has a self-signed cert, the following use of verify=False fails:
import requests
s = requests.Session()
r = s.get('https://selfsignedsite')
_traceback_
r = s.get('https://selfsidnedsite', verify=False)
_traceback is the same about SSL3 not finding a valid cert_
requests.get('https://selfsignedsite', verify=False)
_traceback as above_


fails in what way?

On Sunday, September 28, 2014, Isaul Vargas notifications@github.com
wrote:

> Using a website that has a self-signed cert, the following use of
> verify=False fails:
> import requests
> s = requests.Session()
> r = s.get('https://selfsignedsite')
> 
> r = s.get('https://selfsidnedsite', verify=False)
> 
> requests.get('https://selfsignedsite', verify=False)
> 
> —
> Reply to this email directly or view it on GitHub
> https://github.com/kennethreitz/requests/issues/2255.

## 

## 

Kevin Burke
phone: 925.271.7005 | twentymilliseconds.com


This traceback:

<pre>Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/usr/local/lib/python2.7/site-packages/requests/sessions.py", line 463, in get
    return self.request('GET', url, **kwargs)
  File "/usr/local/lib/python2.7/site-packages/requests/sessions.py", line 451, in request
    resp = self.send(prep, **send_kwargs)
  File "/usr/local/lib/python2.7/site-packages/requests/sessions.py", line 557, in send
    r = adapter.send(request, **kwargs)
  File "/usr/local/lib/python2.7/site-packages/requests/adapters.py", line 420, in send
    raise SSLError(e, request=request)
requests.exceptions.SSLError: [Errno 1] _ssl.c:510: error:14090086:SSL routines:SSL3_GET_SERVER_CERTIFICATE:certificate verify failed</pre>


What version of requests? 


The latest release version 2.4.1


I can't reproduce this at all. Can you share the site you're trying to make a request against?


also does chrome or Firefox load the site?

On Sunday, September 28, 2014, Ian Cordasco notifications@github.com
wrote:

> I can't reproduce this at all. Can you share the site you're trying to
> make a request against?
> 
> —
> Reply to this email directly or view it on GitHub
> https://github.com/kennethreitz/requests/issues/2255#issuecomment-57108647
> .

## 

## 

Kevin Burke
phone: 925.271.7005 | twentymilliseconds.com


Yes firefox and Chrome load the site fine.


The site I was connecting to was <redacted>


Using the regular functional API:

``` pycon
>>> import requests
>>> r = requests.get('https://my.qaa.sailthru-qa.com')
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/usr/local/lib/python2.7/site-packages/requests/api.py", line 59, in get
    return request('get', url, **kwargs)
  File "/usr/local/lib/python2.7/site-packages/requests/api.py", line 48, in request
    return session.request(method=method, url=url, **kwargs)
  File "/usr/local/lib/python2.7/site-packages/requests/sessions.py", line 451, in request
    resp = self.send(prep, **send_kwargs)
  File "/usr/local/lib/python2.7/site-packages/requests/sessions.py", line 557, in send
    r = adapter.send(request, **kwargs)
  File "/usr/local/lib/python2.7/site-packages/requests/adapters.py", line 420, in send
    raise SSLError(e, request=request)
requests.exceptions.SSLError: [Errno 1] _ssl.c:510: error:14090086:SSL routines:SSL3_GET_SERVER_CERTIFICATE:certificate verify failed
>>> r = requests.get('https://my.qaa.sailthru-qa.com', verify=False)
/usr/local/lib/python2.7/site-packages/requests/packages/urllib3/connectionpool.py:730: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.org/en/latest/security.html (This warning will only appear once by default.)
  InsecureRequestWarning)
>>> r.status_code
200
```

Using a Session:

``` pycon
>>> import requests
>>> s = requests.session()
>>> r2 = s.get('https://my.qaa.sailthru-qa.com', verify=False)
/usr/local/lib/python2.7/site-packages/requests/packages/urllib3/connectionpool.py:730: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.org/en/latest/security.html (This warning will only appear once by default.)
  InsecureRequestWarning)
>>> r2.status_code
200
```


In otherwords, we do respect `verify=False`. You may be confused by the warning we emit when you disable certificate verification.


I am not confused by the warning. Just follow the sequence of steps I outlined. Although it is not something I would do in writing the code that I want, I can see instances where if someone writes code to automate connections, and they want to drop back on not using verification, that this would bite them.


Ah, I see what you mean. So this even had bizarre behaviour on 2.3.0:

``` pycon
>>> import requests
>>> s = requests.session()
>>> s.get('https://my.qaa.sailthru-qa.com')
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/Users/icordasc/virtualenv/tmp-a188018336f3351d/lib/python2.7/site-packages/requests/sessions.py", line 468, in get
    return self.request('GET', url, **kwargs)
  File "/Users/icordasc/virtualenv/tmp-a188018336f3351d/lib/python2.7/site-packages/requests/sessions.py", line 456, in request
    resp = self.send(prep, **send_kwargs)
  File "/Users/icordasc/virtualenv/tmp-a188018336f3351d/lib/python2.7/site-packages/requests/sessions.py", line 559, in send
    r = adapter.send(request, **kwargs)
  File "/Users/icordasc/virtualenv/tmp-a188018336f3351d/lib/python2.7/site-packages/requests/adapters.py", line 382, in send
    raise SSLError(e, request=request)
requests.exceptions.SSLError: [Errno 1] _ssl.c:510: error:14090086:SSL routines:SSL3_GET_SERVER_CERTIFICATE:certificate verify failed
>>> s.get('https://my.qaa.sailthru-qa.com', verify=False)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/Users/icordasc/virtualenv/tmp-a188018336f3351d/lib/python2.7/site-packages/requests/sessions.py", line 468, in get
    return self.request('GET', url, **kwargs)
  File "/Users/icordasc/virtualenv/tmp-a188018336f3351d/lib/python2.7/site-packages/requests/sessions.py", line 456, in request
    resp = self.send(prep, **send_kwargs)
  File "/Users/icordasc/virtualenv/tmp-a188018336f3351d/lib/python2.7/site-packages/requests/sessions.py", line 559, in send
    r = adapter.send(request, **kwargs)
  File "/Users/icordasc/virtualenv/tmp-a188018336f3351d/lib/python2.7/site-packages/requests/adapters.py", line 375, in send
    raise ConnectionError(e, request=request)
requests.exceptions.ConnectionError: HTTPSConnectionPool(host='my.qaa.sailthru-qa.com', port=443): Max retries exceeded with url: / (Caused by <class 'httplib.CannotSendRequest'>: )
>>> s.get('https://my.qaa.sailthru-qa.com', verify=False)
<Response [200]>
```

This looks like it might be related to connection pooling. If on 2.4.1 I do this:

``` pycon
>>> s.close()
>>> s.get('https://my.qaa.sailthru-qa.com', verify=False)
<Response [200]>
```

After making the request with `verify=True` you can see that it works just fine (iirc because it clears the pool manager).


Yeah, this is almost certainly connection pooling. urllib3 puts the connection back when SSL errors are raised, but I really don't know if we can safely do that. @shazow, @t-8ch, what are your thoughts about not re-using connections that raise SSL errors?


This should already happen here: https://github.com/kennethreitz/requests/blob/master/requests/packages/urllib3/connectionpool.py#L542


[Not according to my reading](https://github.com/kennethreitz/requests/blob/master/requests/packages/urllib3/connectionpool.py#L566-L570).


Yeah, you are right, but this is the place where it should be done :-)


My question was really whether you think it _should_ be done. =)


Either that, or the connectionpool should include the ssl parameters in the lookup process, so it does not reuse the wrong connection like in this issue.


Hmm, the configuration of sockets in a ConnectionPool should be homogenous. If they're not, then I'd consider this a bug. (That is to say, the configuration of a socket should probably not be affected by request-time flags but rather pool-construction-time flags.)


This appears to be resolved in 2.7.0.