SSL - Connection error

Please refer to our [Stack Overflow tag](https://stackoverflow.com/questions/tagged/python-requests) for guidance.

Hi,
I am writing about this Issue , as all solutions I found so far, did not help to solve my Problem. 
The suggested solutions I tried include: 
1. `https://2.python-requests.org//en/latest/user/advanced/#ssl-cert-verification`
Edit: the sample url: https://requestb.in raises a 404 error actually, probably does not exist even more (?)
2. `https://stackoverflow.com/questions/10667960/python-requests-throwing-sslerror`
3. `https://stackoverflow.com/questions/10667960/python-requests-throwing-sslerror/44543047#44543047`

Nevertheless, most answers have been given at a time, when user used primarly python 2.7 and I am wondering if this is the main problem. 

However. I try to achieve the following goal:

1. reading different urls through a list or a csv file
2. requesting the url, status_code, history, parsed_url.scheme
3. if parsed_url_scheme is http://, than the program should write the url into a csv file, else it should ignore the url. 
Expected outcome: A csv file with only those URLs of the Alexa-Top-100 with http://

My Code so far:
`import requests
from requests.utils import urlparse

```
urls = ['http://www.example.com',
          'http://www.bloomberg.com',
          'https://requestbin.fullcontact.com/'
          'http://www.github.com']

with open('G:\\Request_HEADER_Suite/dummy/http.csv', 'w') as f:
    for url in urls:
        r = requests.get(url, stream=True)
        parsed_url = urlparse(r.url)
        print("URL: ", url)
        print("Redirected to: ", r.url)
        print("Status Code: ", r.status_code)
        print("Scheme: ", parsed_url.scheme)
        if parsed_url.scheme == 'http':
            f.write(url + '\n')`
```
The example above will work fine and as expected. But when I am adding more and more urls for example 20 more URLS at some point I get the following Error:

```
`Traceback (most recent call last):
  File "...\AppData\Local\Programs\Python\Python37\lib\site-packages\urllib3\connection.py", line 160, in _new_conn
    (self._dns_host, self.port), self.timeout, **extra_kw)
  File "...\AppData\Local\Programs\Python\Python37\lib\site-packages\urllib3\util\connection.py", line 57, in create_connection
    for res in socket.getaddrinfo(host, port, family, socket.SOCK_STREAM):
  File "...\AppData\Local\Programs\Python\Python37\lib\socket.py", line 748, in getaddrinfo
    for res in _socket.getaddrinfo(host, port, family, type, proto, flags):
socket.gaierror: [Errno 11001] getaddrinfo failed
```

```
During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "...\AppData\Local\Programs\Python\Python37\lib\site-packages\urllib3\connectionpool.py", line 603, in urlopen
    chunked=chunked)
  File "...\AppData\Local\Programs\Python\Python37\lib\site-packages\urllib3\connectionpool.py", line 355, in _make_request
    conn.request(method, url, **httplib_request_kw)
  File "...\AppData\Local\Programs\Python\Python37\lib\http\client.py", line 1229, in request
    self._send_request(method, url, body, headers, encode_chunked)
  File "...\AppData\Local\Programs\Python\Python37\lib\http\client.py", line 1275, in _send_request
    self.endheaders(body, encode_chunked=encode_chunked)
  File "...\AppData\Local\Programs\Python\Python37\lib\http\client.py", line 1224, in endheaders
    self._send_output(message_body, encode_chunked=encode_chunked)
  File "...\AppData\Local\Programs\Python\Python37\lib\http\client.py", line 1016, in _send_output
    self.send(msg)
  File "...\AppData\Local\Programs\Python\Python37\lib\http\client.py", line 956, in send
    self.connect()
  File "...\AppData\Local\Programs\Python\Python37\lib\site-packages\urllib3\connection.py", line 183, in connect
    conn = self._new_conn()
  File "...\AppData\Local\Programs\Python\Python37\lib\site-packages\urllib3\connection.py", line 169, in _new_conn
    self, "Failed to establish a new connection: %s" % e)
urllib3.exceptions.NewConnectionError: <urllib3.connection.HTTPConnection object at 0x0000023EFC0BBC88>: Failed to establish a new connection: [Errno 11001] getaddrinfo failed

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "...\AppData\Local\Programs\Python\Python37\lib\site-packages\requests\adapters.py", line 449, in send
    timeout=timeout
  File "...\AppData\Local\Programs\Python\Python37\lib\site-packages\urllib3\connectionpool.py", line 641, in urlopen
    _stacktrace=sys.exc_info()[2])
  File "...\AppData\Local\Programs\Python\Python37\lib\site-packages\urllib3\util\retry.py", line 399, in increment
    raise MaxRetryError(_pool, url, error or ResponseError(cause))
urllib3.exceptions.MaxRetryError: HTTPConnectionPool(host='17ok.com', port=80): Max retries exceeded with url: / (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x0000023EFC0BBC88>: Failed to establish a new connection: [Errno 11001] getaddrinfo failed'))

During handling of the above exception, another exception occurred:


Traceback (most recent call last):
  File "...\AppData\Local\Programs\Python\Python37\lib\site-packages\urllib3\connection.py", line 160, in _new_conn
    (self._dns_host, self.port), self.timeout, **extra_kw)
  File "...\AppData\Local\Programs\Python\Python37\lib\site-packages\urllib3\util\connection.py", line 57, in create_connection
    for res in socket.getaddrinfo(host, port, family, socket.SOCK_STREAM):
  File "...\AppData\Local\Programs\Python\Python37\lib\socket.py", line 748, in getaddrinfo
    for res in _socket.getaddrinfo(host, port, family, type, proto, flags):
socket.gaierror: [Errno 11001] getaddrinfo failed

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "...\AppData\Local\Programs\Python\Python37\lib\site-packages\urllib3\connectionpool.py", line 603, in urlopen
    chunked=chunked)
  File "...\AppData\Local\Programs\Python\Python37\lib\site-packages\urllib3\connectionpool.py", line 355, in _make_request
    conn.request(method, url, **httplib_request_kw)
  File "...\AppData\Local\Programs\Python\Python37\lib\http\client.py", line 1229, in request
    self._send_request(method, url, body, headers, encode_chunked)
  File "...\AppData\Local\Programs\Python\Python37\lib\http\client.py", line 1275, in _send_request
    self.endheaders(body, encode_chunked=encode_chunked)
  File "...\AppData\Local\Programs\Python\Python37\lib\http\client.py", line 1224, in endheaders
    self._send_output(message_body, encode_chunked=encode_chunked)
  File "...\AppData\Local\Programs\Python\Python37\lib\http\client.py", line 1016, in _send_output
    self.send(msg)
  File "...\AppData\Local\Programs\Python\Python37\lib\http\client.py", line 956, in send
    self.connect()
  File "...\AppData\Local\Programs\Python\Python37\lib\site-packages\urllib3\connection.py", line 183, in connect
    conn = self._new_conn()
  File "...\AppData\Local\Programs\Python\Python37\lib\site-packages\urllib3\connection.py", line 169, in _new_conn
    self, "Failed to establish a new connection: %s" % e)
urllib3.exceptions.NewConnectionError: <urllib3.connection.HTTPConnection object at 0x0000023EFC0BBC88>: Failed to establish a new connection: [Errno 11001] getaddrinfo failed

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "...\AppData\Local\Programs\Python\Python37\lib\site-packages\requests\adapters.py", line 449, in send
    timeout=timeout
  File "...\AppData\Local\Programs\Python\Python37\lib\site-packages\urllib3\connectionpool.py", line 641, in urlopen
    _stacktrace=sys.exc_info()[2])
  File "...\AppData\Local\Programs\Python\Python37\lib\site-packages\urllib3\util\retry.py", line 399, in increment
    raise MaxRetryError(_pool, url, error or ResponseError(cause))
urllib3.exceptions.MaxRetryError: HTTPConnectionPool(host='17ok.com', port=80): Max retries exceeded with url: / (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x0000023EFC0BBC88>: Failed to establish a new connection: [Errno 11001] getaddrinfo failed'))

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File ".../Request_HEADER_Suite/dummy/dummy-04.py", line 35, in <module>
    r = requests.get(url, stream=True)
  File "...\AppData\Local\Programs\Python\Python37\lib\site-packages\requests\api.py", line 75, in get
    return request('get', url, params=params, **kwargs)
  File "...\AppData\Local\Programs\Python\Python37\lib\site-packages\requests\api.py", line 60, in request
    return session.request(method=method, url=url, **kwargs)
  File "...\AppData\Local\Programs\Python\Python37\lib\site-packages\requests\sessions.py", line 533, in request
    resp = self.send(prep, **send_kwargs)
  File "...\AppData\Local\Programs\Python\Python37\lib\site-packages\requests\sessions.py", line 646, in send
    r = adapter.send(request, **kwargs)
  File "...\AppData\Local\Programs\Python\Python37\lib\site-packages\requests\adapters.py", line 516, in send
    raise ConnectionError(e, request=request)
requests.exceptions.ConnectionError: HTTPConnectionPool(host='17ok.com', port=80): Max retries exceeded with url: / (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x0000023EFC0BBC88>: Failed to establish a new connection: [Errno 11001] getaddrinfo failed'))

```
Process finished with exit code 1
```

And when I am trying to execute the code with the Domains by the Alexa Top-1- Million, I immediately will get an SSL-Error like this:
```

```
Traceback (most recent call last):
  File "...\AppData\Local\Programs\Python\Python37\lib\site-packages\urllib3\connectionpool.py", line 603, in urlopen
    chunked=chunked)
  File "...\AppData\Local\Programs\Python\Python37\lib\site-packages\urllib3\connectionpool.py", line 344, in _make_request
    self._validate_conn(conn)
  File "...\AppData\Local\Programs\Python\Python37\lib\site-packages\urllib3\connectionpool.py", line 843, in _validate_conn
    conn.connect()
  File "...\AppData\Local\Programs\Python\Python37\lib\site-packages\urllib3\connection.py", line 370, in connect
    ssl_context=context)
  File "...\AppData\Local\Programs\Python\Python37\lib\site-packages\urllib3\util\ssl_.py", line 355, in ssl_wrap_socket
    return context.wrap_socket(sock, server_hostname=server_hostname)
  File "...\AppData\Local\Programs\Python\Python37\lib\ssl.py", line 412, in wrap_socket
    session=session
  File "...\AppData\Local\Programs\Python\Python37\lib\ssl.py", line 853, in _create
    self.do_handshake()
  File "...\AppData\Local\Programs\Python\Python37\lib\ssl.py", line 1117, in do_handshake
    self._sslobj.do_handshake()
ssl.SSLError: [SSL: SSLV3_ALERT_HANDSHAKE_FAILURE] sslv3 alert handshake failure (_ssl.c:1056)

During handling of the above exception, another exception occurred:


Traceback (most recent call last):
  File "...\AppData\Local\Programs\Python\Python37\lib\site-packages\requests\adapters.py", line 449, in send
    timeout=timeout
  File "...\AppData\Local\Programs\Python\Python37\lib\site-packages\urllib3\connectionpool.py", line 641, in urlopen
    _stacktrace=sys.exc_info()[2])
  File "...\AppData\Local\Programs\Python\Python37\lib\site-packages\urllib3\util\retry.py", line 399, in increment
    raise MaxRetryError(_pool, url, error or ResponseError(cause))
urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='123movieshub.to', port=443): Max retries exceeded with url: / (Caused by SSLError(SSLError(1, '[SSL: SSLV3_ALERT_HANDSHAKE_FAILURE] sslv3 alert handshake failure (_ssl.c:1056)')))
```


```
During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File ".../Request_HEADER_Suite/dummy/diversifier_http.py", line 506, in <module>
    r = requests.get(url, stream=True)
  File "...\AppData\Local\Programs\Python\Python37\lib\site-packages\requests\api.py", line 75, in get
    return request('get', url, params=params, **kwargs)
  File "...\AppData\Local\Programs\Python\Python37\lib\site-packages\requests\api.py", line 60, in request
    return session.request(method=method, url=url, **kwargs)
  File "...\AppData\Local\Programs\Python\Python37\lib\site-packages\requests\sessions.py", line 533, in request
    resp = self.send(prep, **send_kwargs)
  File "...\AppData\Local\Programs\Python\Python37\lib\site-packages\requests\sessions.py", line 668, in send
    history = [resp for resp in gen] if allow_redirects else []
  File "...\AppData\Local\Programs\Python\Python37\lib\site-packages\requests\sessions.py", line 668, in <listcomp>
    history = [resp for resp in gen] if allow_redirects else []
  File "...\AppData\Local\Programs\Python\Python37\lib\site-packages\requests\sessions.py", line 247, in resolve_redirects
    **adapter_kwargs
  File "...\AppData\Local\Programs\Python\Python37\lib\site-packages\requests\sessions.py", line 646, in send
    r = adapter.send(request, **kwargs)
  File "...\AppData\Local\Programs\Python\Python37\lib\site-packages\requests\adapters.py", line 514, in send
    raise SSLError(e, request=request)
requests.exceptions.SSLError: HTTPSConnectionPool(host='123movieshub.to', port=443): Max retries exceeded with url: / (Caused by SSLError(SSLError(1, '[SSL: SSLV3_ALERT_HANDSHAKE_FAILURE] sslv3 alert handshake failure (_ssl.c:1056)')))

Process finished with exit code 1
```

So, what is the recommended way in 2019 to handle SSL Errors and Connection errors?
Where can I find the most recent solution, as You guys probably have already solved it?
Do you have any demo.py in common for a crawl where you request a list of urls which has either http or https?

I really would appreciate any hints. 
Greetings Felix

---
Env.:
- Win10 and Oracle VIrtual Box - Ubuntu 16.04
- PyCharm Professional 2019








Just for discussing.
From the document, I found this,
> If you set stream to True when making a request, Requests cannot release the connection back to the pool unless you consume all the data or call Response.close. This can lead to inefficiency with connections. If you find yourself partially reading request bodies (or not reading them at all) while using stream=True, you should make the request within a with statement to ensure it’s always closed:

And also, from the exception, it is faild when create a new connection.

If stream=True is not necessaryfor you, I suggest you remove it, and try it again.



Hi @cn-qlg,

thank you, I'll check that. 
Greetings Felix

Hi @felix4webscience,

You're getting SSL errors because the servers you're trying to connect to are using an outdated TLS setup. You can review our `verify` parameter if you must connect to those servers, but I wouldn't advise it. If your requirements are to only connect to http servers, your code can be reworked to avoid these connections.