Request doesnt work on Ubuntu to fetch a TLS1.2 https

I did search long time try to figure out what is the root reason but still have no clue. 
There is a simple script used Request to put information to https://dictation.nuancemobility.net, however, the script works on CentOS box but doesn't on Ubuntu ( both on ElementaryOS Freya which is based on Ubuntu 14.04 TLS and ElementaryOS Luna which is based on Ubuntu 12.02).
I have tried to use adapter, and got an error:

<pre><code>
Traceback (most recent call last):
  File "LanguageIdentify.py", line 11, in <module>
    from APIclasses import speech2text
  File "speech2text.py", line 11, in <module>
    s.mount('https://dictation.nuancemobility.net/', SSLAdapter(ssl.PROTOCOL_TLSv1_2))
AttributeError: 'module' object has no attribute 'PROTOCOL_TLSv1_2'
</code></pre>

I assumed the error caused by python ssl module on Ubuntu is without TLS 1.2 support, but when I ran test on  CentOS box, it got:

<pre><code>
python
\>>>import ssl
>>>'PROTOCOL_TLSv1_2' in dir(ssl)`
False
</code></pre>

So above test informed me at least python ssl in not the root reason.

Some posts show the openssl on Debian/Ubuntu doesnt have default root CA certs configured. So I also tried to follow Certifi doc to import CA certs. But no clue how to operate the next step. I'v tried to follow Request doc that using REQUEST_CA_BUNDLE but also no help.
`export REQUEST_CA_BUNDLE=/etc/ssl/certs/ca-certificates.crt`

Well, all methods I tried and failed on Ubuntu. It is appreciate if anybody can help. 


Can you provide a more detailed description of "doesn't work"? How does it not work? What exception(s) (if any) are you seeing? What version of python are you using? What version of requests?


A quick test locally shows this:

```
$ openssl s_client -connect dictation.nuancemobility.net:443
# ...
SSL-Session:
    Protocol  : TLSv1.2
    Cipher    : RC4-SHA
# ...
$ openssl s_client -no_tls1_2 -connect dictation.nuancemobility.net:443
# ...
SSL-Session:
    Protocol  : TLSv1
    Cipher    : RC4-SHA
# ...
```

However, if we provide a list of ciphers we prefer (as we do in requests)

```
$ openssl s_client -no_tls1_2 -cipher 'ECDH+AESGCM:DH+AESGCM:ECDH+AES256:DH+AES256:ECDH+AES128:DH+AES:ECDH+HIGH:DH+HIGH:ECDH+3DES:DH+3DES:RSA+AESGCM:RSA+AES:RSA+HIGH:RSA+3DES:!aNULL:!eNULL:!MD5' -connect dictation.nuancemobility.net:443
```

Continues to work just fine. We still need much more detail before we can help you debug this and determine if this is _actually_ a requests bug or not (I'm suspecting not).


@sigmavirus24 
Thanks for take care the case.
More details are:

## Environment

1)  A basic ElementaryOS without any configurations, which means a total clean installation from DVD or ISO. (Actually I used 3 different Ubuntu)
- ElementaryOS Freya, based on Ubuntu 14.04 TLS
- ElementaryOS Luna, based on Ubuntu 12.02 TLS
- Docker official python 2.7.10 image which is based on Debian.

2)  A basic CentOS 6.5 without any configuration.
- Python version: 2.6.6
- >>> import ssl
- >>> import requests
- >>> import urllib3

Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
ImportError: No module named urllib3

## Problem

I wrote a script to require Nuance ASR service, see below code. When I run this script on ElementaryOS or Debian, it get a 500 error from Nuance ASR. And Nuance ASR returns correct 200 http response with recognized text content when I run the script on CentOS.

<pre ><code>
import os
import requests
class Nuance_api():
        """Nuance ASR Service API"""
        def __init__(self, appid, appkey, n_id):
                self.appid = appid
                self.appkey = appkey
                self.id = n_id
        def recogenize(self, language, voicefilename):
                nuance_url = 'https://dictation.nuancemobility.net/NMDPAsrCmdServlet/dictation'
                n_params = {'appId':self.appid, 'appKey':self.appkey, 'id':self.id}
                voice_file = {'file': open(voicefilename, 'rb')}
                n_headers = {'Accept-Language':language, 'Content-Type':'audio/amr;codec=amr;bit=4.75;rate=8000', 'Accept':'text/plain', 'Accept-Topic':'Dictation'}
                r = requests.post(url=nuance_url, params=n_params, headers=n_headers, files=voice_file, verify=False)
                if r.status_code == 200 :
                        return r.text
</code></pre>

## Additional problem

When I use `openssl s_client -connect dictation.nuancemobility.net:443` on Ubuntu evn (whatever which one), it gives an error: 

<pre><code>
SSL handshake has read 4032 bytes and written 609 bytes
---
New, TLSv1/SSLv3, Cipher is RC4-SHA
Server public key is 2048 bit
Secure Renegotiation IS supported
Compression: NONE
Expansion: NONE
SSL-Session:
    Protocol  : TLSv1.2
    Cipher    : RC4-SHA
    Session-ID: 9986FC667995B0BEF3325136811A0D8C4E14901627854D8DC042F9920F341903
    Session-ID-ctx: 
    Master-Key: 15199FB75D47C6B7C1CA86D12010C63C34E7908EBC4902EB5AC91E39A6C965121F9906719857CFDCBC319BEA5824C356
    Key-Arg   : None
    PSK identity: None
    PSK identity hint: None
    SRP username: None
    Start Time: 1443494743
    Timeout   : 300 (sec)
    Verify return code: 20 (unable to get local issuer certificate)
---
</code></pre>

When I use `openssl s_client -connect dictation.nuancemobility.net:443` on CentOS env or `openssl s_client -connect dictation.nuancemobility.net:443 -CApath /etc/ssl/certs` on Ubuntu or Debian, it gives correct response:

<pre><code>
---
New, TLSv1/SSLv3, Cipher is RC4-SHA
Server public key is 2048 bit
Secure Renegotiation IS supported
Compression: NONE
Expansion: NONE
SSL-Session:
    Protocol  : TLSv1.2
    Cipher    : RC4-SHA
    Session-ID: 9986FC667994C5F4F3325136811A0C8C0F5A769166086DE7C042F9920F341B05
    Session-ID-ctx: 
    Master-Key: 0B893DBEF29821430934D7AB665AF2E24E6322E7990EB77415A6715D6196F73E503DC0395DDDFF209D491A144371F5D5
    Key-Arg   : None
    Krb5 Principal: None
    PSK identity: None
    PSK identity hint: None
    Start Time: 1443495250
    Timeout   : 300 (sec)
    Verify return code: 0 (ok)
---
</code></pre>

## Reproduce the fault

for debug purpose, I would suggest the easiest way that to create Docker env and run official python 2.7.10 image, then test Request in it. The instructions for these:

<pre><code>
# Docker installation
$ sudo sh -c "echo deb https:/get.docker.io/ubuntu docker main > /etc/apt/sources.list.d/docker.list"
$ curl -s https://get.docker.io/gpg | sudo apt-key add -
$ sudo apt-get update
$ sudo apt-get install lxc-docker
# run docker image
$ sudo docker run -i -t python:2.7.10 /bin/bash
# then the current bash is in python 2.7.10 image, and user is root.
# pip install requests
# python MY_SCRIPT_WITH_NECESSARY_PARAMETERS'_VALUE or just to connect dictation.nuancemobility.net:443
</code></pre>


Looks like your problem is with openssl, not Python, not requests.

You still haven't described the error you're receiving when using requests.


@sigmavirus24 When using Requests, it gets below response.

<pre><code>
/usr/local/lib/python2.7/dist-packages/requests-2.6.0-py2.7.egg/requests/packages/urllib3/connectionpool.py:769: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.org/en/latest/security.html
  InsecureRequestWarning)
<html>
<head>
<meta http-equiv="Content-Type" content="text/html; charset=UTF-8"/>
<title>Error 500 x-nuance-sessionid71887ede-d26d-448d-b154-256eecd3dda0
Received QueryRetry: 1 AUDIO_INFO</title>
</head>
<body><h2>HTTP ERROR 500</h2>
<p>Problem accessing /NMDPAsrCmdServlet/dictation. Reason:
<pre>    x-nuance-sessionid71887ede-d26d-448d-b154-256eecd3dda0
Received QueryRetry: 1 AUDIO_INFO</pre></p>
</body>
</html>
</code></pre>

There are tons discussions of the openssl behavior on Debian/Ubuntu. It is quite clear that every ssl client software on Debian/Ubuntu MUST define CA bundle with some parameters by themselves. The sample is how I use `openssl s_client`. To Requests, I tried `export REQUEST_CA_BUNDLE=/etc/ssl/certs/ca-certificates.crt` but still got 500. 

If Nuance ASR service is hard for troubleshooting. My question can be replaced by "How to make Requests working to access https://www.google.com via TLS v1.2 on Debian/Ubuntu?"

BTW, in order to identify where the problem is, I also tried to revise my code below to use TLS v1, but also got 500. Can you also let me know the usage of Adapter is correct or not please? 

<pre><code>
import os
import requests
from requests_toolbelt.adapters.ssl import SSLAdapter
import ssl
s = requests.Session()
s.mount('https://dictation.nuancemobility.net/', SSLAdapter(ssl.PROTOCOL_TLSv1))
class Nuance_api():
        """Nuance ASR Service API"""
        def __init__(self, appid, appkey, n_id):
                self.appid = appid
                self.appkey = appkey
                self.id = n_id
        def recogenize(self, language, voicefile):
                nuance_url = 'https://dictation.nuancemobility.net/NMDPAsrCmdServlet/dictation'
                n_params = {'appId':self.appid, 'appKey':self.appkey, 'id':self.id}
                voice_file = {'file': open(voicefile, 'rb')}
                n_headers = {'Accept-Language':language, 'Content-Type':'audio/amr;codec=amr;bit=4.75;rate=8000', 'Accept':'text/plain', 'Accept-Topic':'Dictation'}
                r = requests.post(url=nuance_url, params=n_params, headers=n_headers, files=voice_file, verify=False)
                if r.status_code == 200 :
                        return r.text
</code></pre>


A 500 error is a problem with the server. It has nothing to do with TLS or requests.


Also, to answer your question: you are using the SSLAdapter correctly.

To be clearer than I was on my phone:

If you were seeing an exception from requests about an SSLError, that would indicate that the problem was with TLS configuration. The 500 error is something I even see with OpenSSL's `s_client`.

After `s_client` verifies and initiates the connection, you can type anything, e.g., `GET /`. When I do that, the server gives me the following:

```
GET /
<html>
<head>
<meta http-equiv="Content-Type" content="text/html; charset=UTF-8"/>
<title>Error 500 JSP support not configured</title>
</head>
<body><h2>HTTP ERROR 500</h2>
<p>Problem accessing /index.jsp. Reason:
<pre>    JSP support not configured</pre></p>
</body>
</html>
```

This looks to be a problem on their server. Even if I do

```
GET /index.jsp
<html>
<head>
<meta http-equiv="Content-Type" content="text/html; charset=UTF-8"/>
<title>Error 500 JSP support not configured</title>
</head>
<body><h2>HTTP ERROR 500</h2>
<p>Problem accessing /index.jsp. Reason:
<pre>    JSP support not configured</pre></p>
</body>
</html>
```

If I attempt to reproduce your request (a bit):

```
POST /NMDPAsrCmdServlet/dictation
<html>
<head>
<meta http-equiv="Content-Type" content="text/html; charset=UTF-8"/>
<title>Error 401 Unauthorized, invalid application id or key.</title>
</head>
<body><h2>HTTP ERROR 401</h2>
<p>Problem accessing /NMDPAsrCmdServlet/dictation. Reason:
<pre>    Unauthorized, invalid application id or key.</pre></p>
</body>
</html>
```

So their application seems to work up until a point. Then it fails. I'll repeat, this has nothing to do with TLS.


If 500 is caused by server side, why the same code working well on CentOS?

And yes you will get 500 or 401 because there is no necessary credentials in your URL. 

I did further tests in the past hours, on the same Debian/Ubuntu, curl successfully get 200 response, Requests doesn't. So the problem must be caused by Python2 or Requests. 

Catching TCP/IP packages on Debian/Ubuntu, server side send 4 HELLOs after TLS handshaking, but if on CentOS server side will start receiving data after TLS handshaking. 

There is another discussion about SSL failure. Interesting thing in the comment, maybe that is the reason but I don't know how to make Requests working on Debian/Ubuntu.

Discussion:
http://stackoverflow.com/questions/10667960/python-requests-throwing-up-sslerror

comment in this discussion:

> Sorry Boud, I needed to down vote this answer as requests does not handle HTTPS requests "like a web browser". If the full SSL chain of trust (including intermediate certificates) is not declared on a server and requires an extra certificate download, you will receive the above SSL verification error. Web browsers will do the extra download and not flag any certificate errors. This is one way that a web browser and Requests differ. There are others. Requests does some verification, but it is not as good as a browser. –  Louis Cremen Jun 29 at 3:39 


@adrianzhang We're really going to need a working repro scenario to help you here. Can you give us some code that fails in the way you observe where curl succeeds?


@Lukasa 
First of all, I am really appreciate of your support. I did not post the full url because there are credentials in it. Can I send  to you and sigmavirus24 via email? 


@adrianzhang Of course. If you are concerned about security, my GPG key is available at [Keybase](https://keybase.io/lukasa).


@Lukasa Thank you so much, email sent to cory@lukasa.co.uk. Any more details you want to know, feel free and let me know please.


This has been resolved offline. 


Would love to know about the solution... I have the same issue :(


@frenchy75 Are you sure it's the same issue? The issue from the original post was nothing to do with TLS, and it was a coding error in their software. What problem _exactly_ are you seeing?