SNI support seemingly still broken on OS X 10.9

The following does NOT reproduce with Python 3 or on Linux hosts.

Jump to the bottom for the actual error. I just attached logs of the venv setup.

```
bash-3.2$ virtualenv venv
New python executable in venv/bin/python
Installing Setuptools..............................................................................................................................................................................................................................done.
Installing Pip.....................................................................................................................................................................................................................................................................................................................................done.
bash-3.2$ venv/bin/pip install requests urllib3 pyasn1 ndg-httpsclient pyopenssl
Downloading/unpacking requests
  Using download cache from /Users/filosottile/.pip-download-cache/https%3A%2F%2Fpypi.python.org%2Fpackages%2Fsource%2Fr%2Frequests%2Frequests-2.2.1.tar.gz
  Running setup.py egg_info for package requests

Downloading/unpacking urllib3
  Using download cache from /Users/filosottile/.pip-download-cache/https%3A%2F%2Fpypi.python.org%2Fpackages%2Fsource%2Fu%2Furllib3%2Furllib3-1.8.tar.gz
  Running setup.py egg_info for package urllib3

Downloading/unpacking pyasn1
  Using download cache from /Users/filosottile/.pip-download-cache/https%3A%2F%2Fpypi.python.org%2Fpackages%2Fsource%2Fp%2Fpyasn1%2Fpyasn1-0.1.7.tar.gz
  Running setup.py egg_info for package pyasn1

Downloading/unpacking ndg-httpsclient
  Using download cache from /Users/filosottile/.pip-download-cache/https%3A%2F%2Fpypi.python.org%2Fpackages%2Fsource%2Fn%2Fndg-httpsclient%2Fndg_httpsclient-0.3.2.tar.gz
  Running setup.py egg_info for package ndg-httpsclient

Downloading/unpacking pyopenssl
  Using download cache from /Users/filosottile/.pip-download-cache/https%3A%2F%2Fpypi.python.org%2Fpackages%2Fsource%2Fp%2FpyOpenSSL%2FpyOpenSSL-0.14.tar.gz
  Running setup.py egg_info for package pyopenssl

    warning: no previously-included files matching '*.pyc' found anywhere in distribution
    no previously-included directories found matching 'doc/_build'
Downloading/unpacking cryptography>=0.2.1 (from pyopenssl)
  Using download cache from /Users/filosottile/.pip-download-cache/https%3A%2F%2Fpypi.python.org%2Fpackages%2Fsource%2Fc%2Fcryptography%2Fcryptography-0.2.2.tar.gz
  Running setup.py egg_info for package cryptography
    no previously-included directories found matching 'documentation/_build'
    zip_safe flag not set; analyzing archive contents...
    six: module references __file__
    six: module references __path__

    Installed /Users/filosottile/tmp/venv/build/cryptography/six-1.6.1-py2.7.egg
    Searching for cffi>=0.8
    Reading https://pypi.python.org/simple/cffi/
    Best match: cffi 0.8.2
    Downloading https://pypi.python.org/packages/source/c/cffi/cffi-0.8.2.tar.gz#md5=37fc88c62f40d04e8a18192433f951ec
    Processing cffi-0.8.2.tar.gz
    Writing /var/folders/t7/mkv9kpmd4094fb3gfqgkjp640000gn/T/easy_install-zzgElT/cffi-0.8.2/setup.cfg
    Running cffi-0.8.2/setup.py -q bdist_egg --dist-dir /var/folders/t7/mkv9kpmd4094fb3gfqgkjp640000gn/T/easy_install-zzgElT/cffi-0.8.2/egg-dist-tmp-_RWs2A
    Package libffi was not found in the pkg-config search path.
    Perhaps you should add the directory containing `libffi.pc'
    to the PKG_CONFIG_PATH environment variable
    No package 'libffi' found
    Package libffi was not found in the pkg-config search path.
    Perhaps you should add the directory containing `libffi.pc'
    to the PKG_CONFIG_PATH environment variable
    No package 'libffi' found
    Package libffi was not found in the pkg-config search path.
    Perhaps you should add the directory containing `libffi.pc'
    to the PKG_CONFIG_PATH environment variable
    No package 'libffi' found
    Package libffi was not found in the pkg-config search path.
    Perhaps you should add the directory containing `libffi.pc'
    to the PKG_CONFIG_PATH environment variable
    No package 'libffi' found
    Package libffi was not found in the pkg-config search path.
    Perhaps you should add the directory containing `libffi.pc'
    to the PKG_CONFIG_PATH environment variable
    No package 'libffi' found
    OS/X: confusion between 'cc' versus 'gcc' (see issue 123)
    will not use '__thread' in the C code

    Installed /Users/filosottile/tmp/venv/build/cryptography/cffi-0.8.2-py2.7-macosx-10.8-x86_64.egg
    Searching for pycparser
    Reading https://pypi.python.org/simple/pycparser/
    Best match: pycparser 2.10
    Downloading https://pypi.python.org/packages/source/p/pycparser/pycparser-2.10.tar.gz#md5=d87aed98c8a9f386aa56d365fe4d515f
    Processing pycparser-2.10.tar.gz
    Writing /var/folders/t7/mkv9kpmd4094fb3gfqgkjp640000gn/T/easy_install-1JE3Je/pycparser-2.10/setup.cfg
    Running pycparser-2.10/setup.py -q bdist_egg --dist-dir /var/folders/t7/mkv9kpmd4094fb3gfqgkjp640000gn/T/easy_install-1JE3Je/pycparser-2.10/egg-dist-tmp-AjEMEf
    zip_safe flag not set; analyzing archive contents...

    Installed /Users/filosottile/tmp/venv/build/cryptography/pycparser-2.10-py2.7.egg

    building '_cffi__x36bf4a13xfba2f231' extension
    clang -fno-strict-aliasing -fno-common -dynamic -I/usr/local/include -I/usr/local/opt/sqlite/include -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -I/usr/local/Cellar/python/2.7.6/Frameworks/Python.framework/Versions/2.7/include/python2.7 -c cryptography/hazmat/primitives/__pycache__/_cffi__x36bf4a13xfba2f231.c -o /Users/filosottile/tmp/venv/build/cryptography/cryptography/hazmat/primitives/__pycache__/cryptography/hazmat/primitives/__pycache__/_cffi__x36bf4a13xfba2f231.o
    clang -bundle -undefined dynamic_lookup -L/usr/local/lib -L/usr/local/opt/sqlite/lib /Users/filosottile/tmp/venv/build/cryptography/cryptography/hazmat/primitives/__pycache__/cryptography/hazmat/primitives/__pycache__/_cffi__x36bf4a13xfba2f231.o -o /Users/filosottile/tmp/venv/build/cryptography/cryptography/hazmat/primitives/__pycache__/_cffi__x36bf4a13xfba2f231.so
    building '_cffi__xa1394602x4a8b9ec1' extension
    clang -fno-strict-aliasing -fno-common -dynamic -I/usr/local/include -I/usr/local/opt/sqlite/include -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -I/usr/local/Cellar/python/2.7.6/Frameworks/Python.framework/Versions/2.7/include/python2.7 -c cryptography/hazmat/primitives/__pycache__/_cffi__xa1394602x4a8b9ec1.c -o /Users/filosottile/tmp/venv/build/cryptography/cryptography/hazmat/primitives/__pycache__/cryptography/hazmat/primitives/__pycache__/_cffi__xa1394602x4a8b9ec1.o
    clang -bundle -undefined dynamic_lookup -L/usr/local/lib -L/usr/local/opt/sqlite/lib /Users/filosottile/tmp/venv/build/cryptography/cryptography/hazmat/primitives/__pycache__/cryptography/hazmat/primitives/__pycache__/_cffi__xa1394602x4a8b9ec1.o -o /Users/filosottile/tmp/venv/build/cryptography/cryptography/hazmat/primitives/__pycache__/_cffi__xa1394602x4a8b9ec1.so
    building '_cffi__x5eaa210axf0ae7e21' extension
    clang -fno-strict-aliasing -fno-common -dynamic -I/usr/local/include -I/usr/local/opt/sqlite/include -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -I/usr/local/Cellar/python/2.7.6/Frameworks/Python.framework/Versions/2.7/include/python2.7 -c cryptography/hazmat/bindings/__pycache__/_cffi__x5eaa210axf0ae7e21.c -o /Users/filosottile/tmp/venv/build/cryptography/cryptography/hazmat/bindings/__pycache__/cryptography/hazmat/bindings/__pycache__/_cffi__x5eaa210axf0ae7e21.o
    clang -bundle -undefined dynamic_lookup -L/usr/local/lib -L/usr/local/opt/sqlite/lib /Users/filosottile/tmp/venv/build/cryptography/cryptography/hazmat/bindings/__pycache__/cryptography/hazmat/bindings/__pycache__/_cffi__x5eaa210axf0ae7e21.o -lcrypto -lssl -o /Users/filosottile/tmp/venv/build/cryptography/cryptography/hazmat/bindings/__pycache__/_cffi__x5eaa210axf0ae7e21.so
    building '_cffi__x64a6e44fx21ac8a22' extension
    clang -fno-strict-aliasing -fno-common -dynamic -I/usr/local/include -I/usr/local/opt/sqlite/include -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -I/usr/local/Cellar/python/2.7.6/Frameworks/Python.framework/Versions/2.7/include/python2.7 -c cryptography/hazmat/bindings/__pycache__/_cffi__x64a6e44fx21ac8a22.c -o /Users/filosottile/tmp/venv/build/cryptography/cryptography/hazmat/bindings/__pycache__/cryptography/hazmat/bindings/__pycache__/_cffi__x64a6e44fx21ac8a22.o
    clang -bundle -undefined dynamic_lookup -L/usr/local/lib -L/usr/local/opt/sqlite/lib /Users/filosottile/tmp/venv/build/cryptography/cryptography/hazmat/bindings/__pycache__/cryptography/hazmat/bindings/__pycache__/_cffi__x64a6e44fx21ac8a22.o -o /Users/filosottile/tmp/venv/build/cryptography/cryptography/hazmat/bindings/__pycache__/_cffi__x64a6e44fx21ac8a22.so
Downloading/unpacking six>=1.5.2 (from pyopenssl)
  Using download cache from /Users/filosottile/.pip-download-cache/https%3A%2F%2Fpypi.python.org%2Fpackages%2Fsource%2Fs%2Fsix%2Fsix-1.6.1.tar.gz
  Running setup.py egg_info for package six

    no previously-included directories found matching 'documentation/_build'
Downloading/unpacking cffi>=0.8 (from cryptography>=0.2.1->pyopenssl)
  Using download cache from /Users/filosottile/.pip-download-cache/https%3A%2F%2Fpypi.python.org%2Fpackages%2Fsource%2Fc%2Fcffi%2Fcffi-0.8.2.tar.gz
  Running setup.py egg_info for package cffi
    Package libffi was not found in the pkg-config search path.
    Perhaps you should add the directory containing `libffi.pc'
    to the PKG_CONFIG_PATH environment variable
    No package 'libffi' found
    Package libffi was not found in the pkg-config search path.
    Perhaps you should add the directory containing `libffi.pc'
    to the PKG_CONFIG_PATH environment variable
    No package 'libffi' found
    Package libffi was not found in the pkg-config search path.
    Perhaps you should add the directory containing `libffi.pc'
    to the PKG_CONFIG_PATH environment variable
    No package 'libffi' found
    Package libffi was not found in the pkg-config search path.
    Perhaps you should add the directory containing `libffi.pc'
    to the PKG_CONFIG_PATH environment variable
    No package 'libffi' found
    Package libffi was not found in the pkg-config search path.
    Perhaps you should add the directory containing `libffi.pc'
    to the PKG_CONFIG_PATH environment variable
    No package 'libffi' found
    OS/X: confusion between 'cc' versus 'gcc' (see issue 123)
    will not use '__thread' in the C code

Downloading/unpacking pycparser (from cffi>=0.8->cryptography>=0.2.1->pyopenssl)
  Using download cache from /Users/filosottile/.pip-download-cache/https%3A%2F%2Fpypi.python.org%2Fpackages%2Fsource%2Fp%2Fpycparser%2Fpycparser-2.10.tar.gz
  Running setup.py egg_info for package pycparser

Installing collected packages: requests, urllib3, pyasn1, ndg-httpsclient, pyopenssl, cryptography, six, cffi, pycparser
  Running setup.py install for requests

  Running setup.py install for urllib3

  Running setup.py install for pyasn1

  Running setup.py install for ndg-httpsclient
    Skipping installation of /Users/filosottile/tmp/venv/lib/python2.7/site-packages/ndg/__init__.py (namespace package)

    Installing /Users/filosottile/tmp/venv/lib/python2.7/site-packages/ndg_httpsclient-0.3.2-py2.7-nspkg.pth
    Installing ndg_httpclient script to /Users/filosottile/tmp/venv/bin
  Running setup.py install for pyopenssl

    warning: no previously-included files matching '*.pyc' found anywhere in distribution
    no previously-included directories found matching 'doc/_build'
  Running setup.py install for cryptography
    building '_cffi__x5eaa210axf0ae7e21' extension
    clang -fno-strict-aliasing -fno-common -dynamic -I/usr/local/include -I/usr/local/opt/sqlite/include -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -I/usr/local/Cellar/python/2.7.6/Frameworks/Python.framework/Versions/2.7/include/python2.7 -c cryptography/hazmat/bindings/__pycache__/_cffi__x5eaa210axf0ae7e21.c -o build/temp.macosx-10.8-x86_64-2.7/cryptography/hazmat/bindings/__pycache__/_cffi__x5eaa210axf0ae7e21.o
    clang -bundle -undefined dynamic_lookup -L/usr/local/lib -L/usr/local/opt/sqlite/lib build/temp.macosx-10.8-x86_64-2.7/cryptography/hazmat/bindings/__pycache__/_cffi__x5eaa210axf0ae7e21.o -lcrypto -lssl -o build/lib.macosx-10.8-x86_64-2.7/cryptography/_cffi__x5eaa210axf0ae7e21.so
    building '_cffi__x36bf4a13xfba2f231' extension
    clang -fno-strict-aliasing -fno-common -dynamic -I/usr/local/include -I/usr/local/opt/sqlite/include -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -I/usr/local/Cellar/python/2.7.6/Frameworks/Python.framework/Versions/2.7/include/python2.7 -c cryptography/hazmat/primitives/__pycache__/_cffi__x36bf4a13xfba2f231.c -o build/temp.macosx-10.8-x86_64-2.7/cryptography/hazmat/primitives/__pycache__/_cffi__x36bf4a13xfba2f231.o
    clang -bundle -undefined dynamic_lookup -L/usr/local/lib -L/usr/local/opt/sqlite/lib build/temp.macosx-10.8-x86_64-2.7/cryptography/hazmat/primitives/__pycache__/_cffi__x36bf4a13xfba2f231.o -o build/lib.macosx-10.8-x86_64-2.7/cryptography/_cffi__x36bf4a13xfba2f231.so
    building '_cffi__xa1394602x4a8b9ec1' extension
    clang -fno-strict-aliasing -fno-common -dynamic -I/usr/local/include -I/usr/local/opt/sqlite/include -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -I/usr/local/Cellar/python/2.7.6/Frameworks/Python.framework/Versions/2.7/include/python2.7 -c cryptography/hazmat/primitives/__pycache__/_cffi__xa1394602x4a8b9ec1.c -o build/temp.macosx-10.8-x86_64-2.7/cryptography/hazmat/primitives/__pycache__/_cffi__xa1394602x4a8b9ec1.o
    clang -bundle -undefined dynamic_lookup -L/usr/local/lib -L/usr/local/opt/sqlite/lib build/temp.macosx-10.8-x86_64-2.7/cryptography/hazmat/primitives/__pycache__/_cffi__xa1394602x4a8b9ec1.o -o build/lib.macosx-10.8-x86_64-2.7/cryptography/_cffi__xa1394602x4a8b9ec1.so
    building '_cffi__x64a6e44fx21ac8a22' extension
    clang -fno-strict-aliasing -fno-common -dynamic -I/usr/local/include -I/usr/local/opt/sqlite/include -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -I/usr/local/Cellar/python/2.7.6/Frameworks/Python.framework/Versions/2.7/include/python2.7 -c cryptography/hazmat/bindings/__pycache__/_cffi__x64a6e44fx21ac8a22.c -o build/temp.macosx-10.8-x86_64-2.7/cryptography/hazmat/bindings/__pycache__/_cffi__x64a6e44fx21ac8a22.o
    clang -bundle -undefined dynamic_lookup -L/usr/local/lib -L/usr/local/opt/sqlite/lib build/temp.macosx-10.8-x86_64-2.7/cryptography/hazmat/bindings/__pycache__/_cffi__x64a6e44fx21ac8a22.o -o build/lib.macosx-10.8-x86_64-2.7/cryptography/_cffi__x64a6e44fx21ac8a22.so

  Running setup.py install for six

    no previously-included directories found matching 'documentation/_build'
  Running setup.py install for cffi
    Package libffi was not found in the pkg-config search path.
    Perhaps you should add the directory containing `libffi.pc'
    to the PKG_CONFIG_PATH environment variable
    No package 'libffi' found
    Package libffi was not found in the pkg-config search path.
    Perhaps you should add the directory containing `libffi.pc'
    to the PKG_CONFIG_PATH environment variable
    No package 'libffi' found
    Package libffi was not found in the pkg-config search path.
    Perhaps you should add the directory containing `libffi.pc'
    to the PKG_CONFIG_PATH environment variable
    No package 'libffi' found
    Package libffi was not found in the pkg-config search path.
    Perhaps you should add the directory containing `libffi.pc'
    to the PKG_CONFIG_PATH environment variable
    No package 'libffi' found
    Package libffi was not found in the pkg-config search path.
    Perhaps you should add the directory containing `libffi.pc'
    to the PKG_CONFIG_PATH environment variable
    No package 'libffi' found
    OS/X: confusion between 'cc' versus 'gcc' (see issue 123)
    will not use '__thread' in the C code
    building '_cffi_backend' extension
    clang -fno-strict-aliasing -fno-common -dynamic -I/usr/local/include -I/usr/local/opt/sqlite/include -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -I/usr/include/ffi -I/usr/include/libffi -I/usr/local/Cellar/python/2.7.6/Frameworks/Python.framework/Versions/2.7/include/python2.7 -c c/_cffi_backend.c -o build/temp.macosx-10.8-x86_64-2.7/c/_cffi_backend.o
    clang -bundle -undefined dynamic_lookup -L/usr/local/lib -L/usr/local/opt/sqlite/lib build/temp.macosx-10.8-x86_64-2.7/c/_cffi_backend.o -lffi -o build/lib.macosx-10.8-x86_64-2.7/_cffi_backend.so

  Running setup.py install for pycparser

Successfully installed requests urllib3 pyasn1 ndg-httpsclient pyopenssl cryptography six cffi pycparser
Cleaning up...
bash-3.2$ venv/bin/pip freeze
cffi==0.8.2
cryptography==0.2.2
ndg-httpsclient==0.3.2
pyOpenSSL==0.14
pyasn1==0.1.7
pycparser==2.10
requests==2.2.1
six==1.6.1
urllib3==1.8
wsgiref==0.1.2
bash-3.2$ venv/bin/python
Python 2.7.6 (default, Dec  4 2013, 18:45:48)
[GCC 4.2.1 Compatible Apple LLVM 5.0 (clang-500.2.75)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> import requests
>>> requests.get("https://transport03-rts03-iad01.transport.home.nest.com:443/")
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/Users/filosottile/tmp/venv/lib/python2.7/site-packages/requests/api.py", line 55, in get
    return request('get', url, **kwargs)
  File "/Users/filosottile/tmp/venv/lib/python2.7/site-packages/requests/api.py", line 44, in request
    return session.request(method=method, url=url, **kwargs)
  File "/Users/filosottile/tmp/venv/lib/python2.7/site-packages/requests/sessions.py", line 383, in request
    resp = self.send(prep, **send_kwargs)
  File "/Users/filosottile/tmp/venv/lib/python2.7/site-packages/requests/sessions.py", line 486, in send
    r = adapter.send(request, **kwargs)
  File "/Users/filosottile/tmp/venv/lib/python2.7/site-packages/requests/adapters.py", line 385, in send
    raise SSLError(e)
requests.exceptions.SSLError: [Errno bad handshake] (-1, 'Unexpected EOF')
>>> 
```


I don't think this has anything to do with SNI, and instead has everything to do with the OpenSSL that ships on OS X being braindead and stupid. I've run this on 2.7.6 and 3.3.4, both on OS X. In both cases, the SNI extension is being correctly sent to the remote server. The difference is that the server, in the Python 2.7.6 case, is not responding to the TLS Client Hello.

There could be a number of reasons for this. In Python 2.7.6 we do a TLS v1 handshake, but in Python 3.3.4 we do a TLSv1.2 handshake. This is unlikely, as the server negotiates down to TLSv1. More likely, the server doesn't like the cipher suites we offer. In 3.3.4 we offer 72 cipher suites, and the server chooses TLS_DHE_RSA_WITH_3DES_EDE_CBC_SHA. In 2.7.6 we offer 2 cipher suites: TLS_RSA_WITH_RC4_128_SHA and TLS_EMPTY_RENEGOTIATION_INFO_SCSV. If the server in this case is expecting to choose a suite we've offered, but doesn't allow TLS_RSA_WITH_RC4_128_SHA (which is pretty weak), it will find that there's nothing for it to choose.

Either way, we're sending SNI and it's right, so the problem is not requests. =)


The analysis makes sense. Sadly linking Python against a better (2014) OpenSSL didn't fix it.

I'll have to dig deeper. Thanks!


This is happening to me on CentOS 5 as well.


@mattwilliamson _What specifically_ is happening to you?


@Lukasa clearly _it_ is happening. :wink: