Repository: conda/conda

Issue:
OpenSSL.SSL.Error while creating new environment
Hi,

I am trying to create a new conda environment but getting the following error:
`$ /opt/external/conda/bin/conda env create -n aaa -vvv
Fetching package metadata ...An unexpected error has occurred.
Please consider posting the following information to the
conda GitHub issue tracker at:

    https://github.com/conda/conda/issues



Current conda install:

               platform : linux-64
          conda version : 4.3.16
       conda is private : False
      conda-env version : 4.3.16
    conda-build version : not installed
         python version : 3.6.0.final.0
       requests version : 2.12.4
       root environment : /opt/external/conda  (read only)
    default environment : /opt/external/conda
       envs directories : /opt/apps/adm-pbak
                          /home/adm-pbak/.conda/envs
                          /opt/external/conda/envs
          package cache : /opt/external/conda/pkgs
                          /opt/apps/adm-pbak/.conda/pkgs
           channel URLs : https://repo.continuum.io/pkgs/free/linux-64
                          https://repo.continuum.io/pkgs/free/noarch
                          https://repo.continuum.io/pkgs/r/linux-64
                          https://repo.continuum.io/pkgs/r/noarch
                          https://repo.continuum.io/pkgs/pro/linux-64
                          https://repo.continuum.io/pkgs/pro/noarch
            config file : /home/adm-pbak/.condarc
           offline mode : False
             user-agent : conda/4.3.16 requests/2.12.4 CPython/3.6.0 Linux/2.6.32-642.13.1.el6.x86_64 Red Hat Enterprise Linux Server/6.8 glibc/2.12
                UID:GID : 34002:34000



`$ /opt/external/conda/bin/conda-env create -n aaa -vvv`




    Traceback (most recent call last):
      File "/opt/external/conda/lib/python3.6/site-packages/conda/exceptions.py", line 626, in conda_exception_handler
        return_value = func(*args, **kwargs)
      File "/opt/external/conda/lib/python3.6/site-packages/conda_env/cli/main_create.py", line 108, in execute
        installer.install(prefix, pkg_specs, args, env)
      File "/opt/external/conda/lib/python3.6/site-packages/conda_env/installers/conda.py", line 29, in install
        prefix=prefix)
      File "/opt/external/conda/lib/python3.6/site-packages/conda/core/index.py", line 92, in get_index
        index = fetch_index(channel_priority_map, use_cache=use_cache)
      File "/opt/external/conda/lib/python3.6/site-packages/conda/core/index.py", line 111, in fetch_index
        repodatas = collect_all_repodata(use_cache, tasks)
      File "/opt/external/conda/lib/python3.6/site-packages/conda/core/repodata.py", line 74, in collect_all_repodata
        repodatas = _collect_repodatas_serial(use_cache, tasks)
      File "/opt/external/conda/lib/python3.6/site-packages/conda/core/repodata.py", line 463, in _collect_repodatas_serial
        for url, schan, pri in tasks]
      File "/opt/external/conda/lib/python3.6/site-packages/conda/core/repodata.py", line 463, in <listcomp>
        for url, schan, pri in tasks]
      File "/opt/external/conda/lib/python3.6/site-packages/conda/core/repodata.py", line 109, in func
        res = f(*args, **kwargs)
      File "/opt/external/conda/lib/python3.6/site-packages/conda/core/repodata.py", line 442, in fetch_repodata
        mod_etag_headers.get('_mod'))
      File "/opt/external/conda/lib/python3.6/site-packages/conda/core/repodata.py", line 138, in fetch_repodata_remote_request
        timeout=timeout)
      File "/opt/external/conda/lib/python3.6/site-packages/requests/sessions.py", line 501, in get
        return self.request('GET', url, **kwargs)
      File "/opt/external/conda/lib/python3.6/site-packages/requests/sessions.py", line 488, in request
        resp = self.send(prep, **send_kwargs)
      File "/opt/external/conda/lib/python3.6/site-packages/requests/sessions.py", line 609, in send
        r = adapter.send(request, **kwargs)
      File "/opt/external/conda/lib/python3.6/site-packages/requests/adapters.py", line 423, in send
        timeout=timeout
      File "/opt/external/conda/lib/python3.6/site-packages/requests/packages/urllib3/connectionpool.py", line 588, in urlopen
        self._prepare_proxy(conn)
      File "/opt/external/conda/lib/python3.6/site-packages/requests/packages/urllib3/connectionpool.py", line 801, in _prepare_proxy
        conn.connect()
      File "/opt/external/conda/lib/python3.6/site-packages/requests/packages/urllib3/connection.py", line 323, in connect
        ssl_context=context)
      File "/opt/external/conda/lib/python3.6/site-packages/requests/packages/urllib3/util/ssl_.py", line 322, in ssl_wrap_socket
        context.load_cert_chain(certfile, keyfile)
      File "/opt/external/conda/lib/python3.6/site-packages/requests/packages/urllib3/contrib/pyopenssl.py", line 397, in load_cert_chain
        self._ctx.use_certificate_file(certfile)
      File "/opt/external/conda/lib/python3.6/site-packages/OpenSSL/SSL.py", line 596, in use_certificate_file
        _raise_current_error()
      File "/opt/external/conda/lib/python3.6/site-packages/OpenSSL/_util.py", line 48, in exception_from_error_queue
        raise exception_type(errors)
    OpenSSL.SSL.Error: [('system library', 'fopen', 'No such file or directory'), ('BIO routines', 'FILE_CTRL', 'system lib'), ('SSL routines', 'SSL_CTX_use_certificate_file', 'system lib')]

`
And this is my .condarc file:
`add_anaconda_token: True
add_pip_as_python_dependency: True
allow_non_channel_urls: True
allow_softlinks: True
always_copy: False
always_softlink: False
always_yes: False
anaconda_upload: None
auto_update_conda: True
changeps1: True
channel_alias: https://conda.anaconda.org
channel_priority: True
channels:
  - defaults
client_ssl_cert:
client_ssl_cert_key:
clobber: False
create_default_packages: []
custom_channels:
  pkgs/free: https://repo.continuum.io/
  pkgs/r: https://repo.continuum.io/
  pkgs/pro: https://repo.continuum.io/
custom_multichannels:
  defaults: ["https://repo.continuum.io/pkgs/free", "https://repo.continuum.io/pkgs/r", "https://repo.continuum.io/pkgs/pro"]
  local: []
default_channels:
  - https://repo.continuum.io/pkgs/free
  - https://repo.continuum.io/pkgs/r
  - https://repo.continuum.io/pkgs/pro
disallow: []
envs_dirs:
  - /opt/apps/adm-pbak
  - /home/adm-pbak/.conda/envs
  - /opt/external/conda/envs
force: False
json: False
local_repodata_ttl: 1
migrated_channel_aliases: []
offline: False
path_conflict: clobber
pinned_packages: []
pkgs_dirs:
  - /opt/external/conda/pkgs
  - /opt/apps/adm-pbak/.conda/pkgs
proxy_servers: {}
quiet: False
remote_connect_timeout_secs: 9.15
remote_max_retries: 3
remote_read_timeout_secs: 60.0
rollback_enabled: True
shortcuts: True
show_channel_urls: None
ssl_verify: False
track_features: []
use_pip: True
verbosity: 0
`


Discussion:
@przemolb Do you have the environment variable `REQUESTS_CA_BUNDLE` set? What is the output for:
```
env | grep -i CA_BUNDLE
```
If it is set, make sure that the file which is present in the value, exists, and is up to date, otherwise unset this environment variable by typing:
```
unset REQUESTS_CA_BUNDLE
```
And then try creating a conda environment.
I don't have it set:
`
[adm-pbak@dsvuk3lifm001 ~]$ env | grep -i CA_BUNDLE
[adm-pbak@dsvuk3lifm001 ~]$ unset REQUESTS_CA_BUNDLE
[adm-pbak@dsvuk3lifm001 ~]$ /opt/external/conda/bin/conda env create -f environment.yml -p /opt/apps/adm-pbak/
Fetching package metadata ...An unexpected error has occurred.
Please consider posting the following information to the
conda GitHub issue tracker at:

    https://github.com/conda/conda/issues



Current conda install:

               platform : linux-64
          conda version : 4.3.16
       conda is private : False
      conda-env version : 4.3.16
    conda-build version : not installed
         python version : 3.6.1.final.0
       requests version : 2.13.0
       root environment : /opt/external/conda  (read only)
    default environment : /opt/external/conda
       envs directories : /opt/apps/adm-pbak
                          /home/adm-pbak/.conda/envs
                          /opt/external/conda/envs
          package cache : /opt/apps/adm-pbak
                          /opt/external/conda/pkgs
                          /opt/apps/adm-pbak/.conda/pkgs
           channel URLs : https://repo.continuum.io/pkgs/free/linux-64
                          https://repo.continuum.io/pkgs/free/noarch
                          https://repo.continuum.io/pkgs/r/linux-64
                          https://repo.continuum.io/pkgs/r/noarch
                          https://repo.continuum.io/pkgs/pro/linux-64
                          https://repo.continuum.io/pkgs/pro/noarch
            config file : /home/adm-pbak/.condarc
           offline mode : False
             user-agent : conda/4.3.16 requests/2.13.0 CPython/3.6.1 Linux/2.6.32-642.13.1.el6.x86_64 Red Hat Enterprise Linux Server/6.8 glibc/2.12
                UID:GID : 34002:34000



`$ /opt/external/conda/bin/conda-env create -f environment.yml -p /opt/apps/adm-pbak/`




    Traceback (most recent call last):
      File "/opt/external/conda/lib/python3.6/site-packages/conda/exceptions.py", line 626, in conda_exception_handler
        return_value = func(*args, **kwargs)
      File "/opt/external/conda/lib/python3.6/site-packages/conda_env/cli/main_create.py", line 108, in execute
        installer.install(prefix, pkg_specs, args, env)
      File "/opt/external/conda/lib/python3.6/site-packages/conda_env/installers/conda.py", line 29, in install
        prefix=prefix)
      File "/opt/external/conda/lib/python3.6/site-packages/conda/core/index.py", line 92, in get_index
        index = fetch_index(channel_priority_map, use_cache=use_cache)
      File "/opt/external/conda/lib/python3.6/site-packages/conda/core/index.py", line 111, in fetch_index
        repodatas = collect_all_repodata(use_cache, tasks)
      File "/opt/external/conda/lib/python3.6/site-packages/conda/core/repodata.py", line 74, in collect_all_repodata
        repodatas = _collect_repodatas_serial(use_cache, tasks)
      File "/opt/external/conda/lib/python3.6/site-packages/conda/core/repodata.py", line 463, in _collect_repodatas_serial
        for url, schan, pri in tasks]
      File "/opt/external/conda/lib/python3.6/site-packages/conda/core/repodata.py", line 463, in <listcomp>
        for url, schan, pri in tasks]
      File "/opt/external/conda/lib/python3.6/site-packages/conda/core/repodata.py", line 109, in func
        res = f(*args, **kwargs)
      File "/opt/external/conda/lib/python3.6/site-packages/conda/core/repodata.py", line 442, in fetch_repodata
        mod_etag_headers.get('_mod'))
      File "/opt/external/conda/lib/python3.6/site-packages/conda/core/repodata.py", line 138, in fetch_repodata_remote_request
        timeout=timeout)
      File "/opt/external/conda/lib/python3.6/site-packages/requests/sessions.py", line 501, in get
        return self.request('GET', url, **kwargs)
      File "/opt/external/conda/lib/python3.6/site-packages/requests/sessions.py", line 488, in request
        resp = self.send(prep, **send_kwargs)
      File "/opt/external/conda/lib/python3.6/site-packages/requests/sessions.py", line 609, in send
        r = adapter.send(request, **kwargs)
      File "/opt/external/conda/lib/python3.6/site-packages/requests/adapters.py", line 423, in send
        timeout=timeout
      File "/opt/external/conda/lib/python3.6/site-packages/requests/packages/urllib3/connectionpool.py", line 594, in urlopen
        self._prepare_proxy(conn)
      File "/opt/external/conda/lib/python3.6/site-packages/requests/packages/urllib3/connectionpool.py", line 810, in _prepare_proxy
        conn.connect()
      File "/opt/external/conda/lib/python3.6/site-packages/requests/packages/urllib3/connection.py", line 326, in connect
        ssl_context=context)
      File "/opt/external/conda/lib/python3.6/site-packages/requests/packages/urllib3/util/ssl_.py", line 322, in ssl_wrap_socket
        context.load_cert_chain(certfile, keyfile)
      File "/opt/external/conda/lib/python3.6/site-packages/requests/packages/urllib3/contrib/pyopenssl.py", line 416, in load_cert_chain
        self._ctx.use_certificate_file(certfile)
      File "/opt/external/conda/lib/python3.6/site-packages/OpenSSL/SSL.py", line 596, in use_certificate_file
        _raise_current_error()
      File "/opt/external/conda/lib/python3.6/site-packages/OpenSSL/_util.py", line 48, in exception_from_error_queue
        raise exception_type(errors)
    OpenSSL.SSL.Error: [('system library', 'fopen', 'No such file or directory'), ('BIO routines', 'FILE_CTRL', 'system lib'), ('SSL routines', 'SSL_CTX_use_certificate_file', 'system lib')]

`
@przemolb Could you please share the output for the command:
```bash
conda config --show | grep client_ssl_cert
```
If it is set to some file location, could you please check whether the file exists?
It appears as if this is some sort of centralized conda installation with some pre-configured settings for proxy and ssl certificates. Could you also please share the output for:
```
find /opt/external/conda/ -name .condarc
```
[adm-pbak@dsvuk3lifm001 ~]$ /opt/external/conda/bin/conda config --show | grep client_ssl_cert          
client_ssl_cert: None
client_ssl_cert_key: None

`
[adm-pbak@dsvuk3lifm001 ~]$ find /opt/external/conda/ -name .condarc
[adm-pbak@dsvuk3lifm001 ~]$ ls -la .condarc*
-rw-r--r-- 1 adm-pbak adm-sysadmin 1392 Apr 19 10:19 .condarc
[adm-pbak@dsvuk3lifm001 ~]$ cat .condarc 
add_anaconda_token: True
add_pip_as_python_dependency: True
allow_non_channel_urls: True
allow_softlinks: True
always_copy: False
always_softlink: False
always_yes: False
anaconda_upload: None
auto_update_conda: True
changeps1: True
channel_alias: https://conda.anaconda.org
channel_priority: True
channels:
  - defaults
client_ssl_cert: 
client_ssl_cert_key: 
clobber: False
create_default_packages: []
custom_channels:
  pkgs/free: https://repo.continuum.io/
  pkgs/r: https://repo.continuum.io/
  pkgs/pro: https://repo.continuum.io/
custom_multichannels:
  defaults: ["https://repo.continuum.io/pkgs/free", "https://repo.continuum.io/pkgs/r", "https://repo.continuum.io/pkgs/pro"]
  local: []
default_channels:
  - https://repo.continuum.io/pkgs/free
  - https://repo.continuum.io/pkgs/r
  - https://repo.continuum.io/pkgs/pro
disallow: []
envs_dirs:
  - /opt/apps/adm-pbak
  - /home/adm-pbak/.conda/envs
  - /opt/external/conda/envs
force: False
json: False
local_repodata_ttl: 1
migrated_channel_aliases: []
offline: False
path_conflict: clobber
pinned_packages: []
pkgs_dirs:
  - /opt/external/conda/pkgs
  - /opt/apps/adm-pbak/.conda/pkgs
proxy_servers: {}
quiet: False
remote_connect_timeout_secs: 9.15
remote_max_retries: 3
remote_read_timeout_secs: 60.0
rollback_enabled: True
shortcuts: True
show_channel_urls: None
ssl_verify: False
track_features: []
use_pip: True
verbosity: 0
`
@przemolb Ah, I think now I am able to reproduce your scenario. Could you please try the following? Edit the file `/home/adm-pbak/.condarc` and remove the following two lines:
```
client_ssl_cert:
client_ssl_cert_key:
```
FYI, `conda config --show-sources` could have also been helpful in diagnosing :)
`conda config --show` and `conda config --show-sources` are slightly different, and both are useful
@kalefranz  I am not sure if this is the right solution, but the following patch fixes this issue for me:
```patch
diff --git a/conda/base/context.py b/conda/base/context.py
index 78033d6..46e9138 100644
--- a/conda/base/context.py
+++ b/conda/base/context.py
@@ -103,8 +102,10 @@ class Context(Configuration):
     # remote connection details
     ssl_verify = PrimitiveParameter(True, element_type=string_types + (bool,),
                                     validation=ssl_verify_validation)
-    client_ssl_cert = PrimitiveParameter('', aliases=('client_cert',))
-    client_ssl_cert_key = PrimitiveParameter('', aliases=('client_cert_key',))
+    client_ssl_cert = PrimitiveParameter(None, aliases=('client_cert',),
+                                         element_type=string_types + (NoneType,))
+    client_ssl_cert_key = PrimitiveParameter(None, aliases=('client_cert_key',),
+                                             element_type=string_types + (NoneType,))
```
As requested:
`
[adm-pbak@dsvuk3lifm001 ~]$ /opt/external/conda/bin/conda config --show-sources                                                                                                                                    
==> /home/adm-pbak/.condarc <==                                                                                                                                                                                                              
add_pip_as_python_dependency: True
allow_softlinks: True
auto_update_conda: True
clobber: False
changeps1: True
create_default_packages: []
disallow: []
path_conflict: clobber
pinned_packages: []
rollback_enabled: True
track_features: []
use_pip: True
envs_dirs:
  - /opt/apps/adm-pbak
  - /home/adm-pbak/.conda/envs
  - /opt/external/conda/envs
pkgs_dirs:
  - /opt/external/conda/pkgs
  - /opt/apps/adm-pbak/.conda/pkgs
local_repodata_ttl: 1
ssl_verify: False
client_ssl_cert: None
client_ssl_cert_key: None
proxy_servers: {}
remote_connect_timeout_secs: 9.15
remote_read_timeout_secs: 60.0
remote_max_retries: 3
add_anaconda_token: True
channel_alias: https://conda.anaconda.org
allow_non_channel_urls: True
channels:
  - defaults
migrated_channel_aliases: []
default_channels:
  - https://repo.continuum.io/pkgs/free
  - https://repo.continuum.io/pkgs/r
  - https://repo.continuum.io/pkgs/pro
custom_channels:
  pkgs/free: https://repo.continuum.io/
  pkgs/r: https://repo.continuum.io/
  pkgs/pro: https://repo.continuum.io/
custom_multichannels:
  defaults: ['https://repo.continuum.io/pkgs/free', 'https://repo.continuum.io/pkgs/r', 'https://repo.continuum.io/pkgs/pro']
  local: []
always_softlink: False
always_copy: False
always_yes: False
channel_priority: True
force: False
json: False
offline: False
quiet: False
shortcuts: True
show_channel_urls: None
verbosity: 0
anaconda_upload: None

==> envvars <==
envs_path:
  - /opt/apps/adm-pbak/
pkgs_dirs:
  - /opt/apps/adm-pbak/

`
@przemolb The problem is that:

```
client_ssl_cert: None
client_ssl_cert_key: None
```
Is literally making `client_ssl_cert` equal to the string `None`, whereas it should be interpreted as 'this is not set', hence I asked you to remove both of these strings from `~/.condarc` and try to run the `conda-env create ...` command
`
[adm-pbak@dsvuk3lifm001 ~]$ vim .condarc  # removed client_ssl_cert and client_ssl_cert_key from the config
[adm-pbak@dsvuk3lifm001 ~]$ /opt/external/conda/bin/conda env create -f environment.yml -p /opt/apps/adm-pbak/
Fetching package metadata .........
Solving package specifications: .
readline-6.2-2 100% |#############################################################################################################################################################################################| Time: 0:00:00  12.21 MB/s
sqlite-3.13.0- 100% |#############################################################################################################################################################################################| Time: 0:00:00  31.13 MB/s
tk-8.5.18-0.ta 100% |#############################################################################################################################################################################################| Time: 0:00:00  36.14 MB/s
xz-5.2.2-1.tar 100% |#############################################################################################################################################################################################| Time: 0:00:00  28.18 MB/s
zlib-1.2.8-3.t 100% |#############################################################################################################################################################################################| Time: 0:00:00  26.52 MB/s
setuptools-27. 100% |#############################################################################################################################################################################################| Time: 0:00:00  31.03 MB/s
wheel-0.29.0-p 100% |#############################################################################################################################################################################################| Time: 0:00:00  24.61 MB/s
pip-9.0.1-py36 100% |#############################################################################################################################################################################################| Time: 0:00:00  31.76 MB/s
#
# To activate this environment, use:
# > source activate /opt/apps/adm-pbak
#
# To deactivate this environment, use:
# > source deactivate /opt/apps/adm-pbak
#
`

So now it is working fine :-)
When we can expect the patch to be released ?

Wow it certainly could be. Makes sense to me.  Want to submit a PR targeted at 4.3.x?  I'm getting ready to tag 4.3.17 in the next day or two.  Could easily get it in.

Right now we don't have proxy servers and such to integration test a lot of these http connection issues.  We need them.  It's on the TODO list.
I'll perform some tests locally and send a PR targeting 4.3.x

Resolving patch:
diff --git a/conda/base/context.py b/conda/base/context.py
--- a/conda/base/context.py
+++ b/conda/base/context.py
@@ -103,8 +103,10 @@ class Context(Configuration):
     # remote connection details
     ssl_verify = PrimitiveParameter(True, element_type=string_types + (bool,),
                                     validation=ssl_verify_validation)
-    client_ssl_cert = PrimitiveParameter('', aliases=('client_cert',))
-    client_ssl_cert_key = PrimitiveParameter('', aliases=('client_cert_key',))
+    client_ssl_cert = PrimitiveParameter(None, aliases=('client_cert',),
+                                         element_type=string_types + (NoneType,))
+    client_ssl_cert_key = PrimitiveParameter(None, aliases=('client_cert_key',),
+                                             element_type=string_types + (NoneType,))
     proxy_servers = MapParameter(string_types)
     remote_connect_timeout_secs = PrimitiveParameter(9.15)
     remote_read_timeout_secs = PrimitiveParameter(60.)


Test patch: