Repository: conan-io/conan

Issue:
conan config install https://... fails when conan_server's certificate is self-signed
Hello,

I am using conan 1.0.4 and experienced an issue while I was trying to install a configuration for a conan client ([\[docs.conan.io\] conan config install](http://docs.conan.io/en/latest/reference/commands/consumer/config.html)).
This server has a self-signed certificate.
```
$ conan config install https://conan_server/conan-client/config/conan-default-config.zip
Trying to download  https://conan_server/conan-client/config/conan-default-config.zip
ERROR: Error downloading file https://conan_server/conan-client/config/conan-default-config.zip: 'HTTPSConnectionPool(host='conan_server', port=443): Max retries exceeded with url: /conan-client/config/conan-default-config.zip (Caused by SSLError(SSLError("bad handshake: Error([('SSL routines', 'tls_process_server_certificate', 'certificate verify failed')],)",),))'
Waiting 5 seconds to retry...
ERROR: Error downloading file https://conan_server/conan-client/config/conan-default-config.zip: 'HTTPSConnectionPool(host='conan_server', port=443): Max retries exceeded with url: /conan-client/config/conan-default-config.zip (Caused by SSLError(SSLError("bad handshake: Error([('SSL routines', 'tls_process_server_certificate', 'certificate verify failed')],)",),))'
```
`conan config install` does not have currently an option to disable the verification of the certificate.
The download is performed in [conans/client/conf/config_installer.py#L83](https://github.com/conan-io/conan/blob/8c0b516f3a13efe68144d146d86b9ba80c213624/conans/client/conf/config_installer.py#L83) using `tools.download()`. This utility has a parameter to enable/disable ssl checks.
```python
def download(url, filename, verify=True, out=None, retry=2, retry_wait=5, overwrite=False,
             auth=None, headers=None)
```
I think a flag to disable ssl check would be a good solution in this use case.
What do you think about it ?
Is there currently another solution to perform the download from a server with a self-signed certificate ?

EDIT:
Someone running a git repository with a sefl-signed certificate will experience the same issue.
```python
def _process_git_repo(repo_url, client_cache, output, runner, tmp_folder):
    output.info("Trying to clone repo  %s" % repo_url)

    with tools.chdir(tmp_folder):
        runner('git clone "%s" config' % repo_url, output=output)
    tmp_folder = os.path.join(tmp_folder, "config")
    _process_folder(tmp_folder, client_cache, output)
```
I guess this case can be solved by setting manually `GIT_SSL_NO_VERIFY` to `true`.


Discussion:
Hi!

Thanks for reporting this issue, indeed it seems a problem.

Could you please try hardcoding ``verify=False`` in https://github.com/conan-io/conan/blob/8c0b516f3a13efe68144d146d86b9ba80c213624/conans/client/conf/config_installer.py#L83, to check that it really solves your issue? Then we could think of exposing that as an argument.

Another approach would be trying to add your own self-signed certificate to the ``<userhome>/.conan/cacert.pem`` file. I understand this is manual, and then not very convenient, but would keep the SSL checks active. 
Hardcoding `verify=False` worked.
```
$ conan config install https://conan_server/config
Trying to download  https://conan_server/config
/usr/local/lib/python3.5/dist-packages/urllib3/connectionpool.py:858: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings
  InsecureRequestWarning)
[==================================================] 2.9KB/2.9KB      
Unzipping 5.0KB
Unzipping 100 %                                                       Installing settings.yml
Defining remotes
WARN: Remotes registry file missing, creating default one in /home/matt/.conan/registry.txt
Installing profiles
    Installing profile rhel-gcc49
Processing conan.conf
```
Ok, thanks for telling, looks good. We will have a look and discuss the UI for this opt-in, I guess something like ``--verify=False`` in command line, though it looks a bit ugly. Lets think about it.

Would be the other approach viable for you? Adding your certificate to the ``cacert.pem`` file? Thanks!
I am not in favor of `cacert.pem` because the user has to perform one more operation to download and install this file whereas the purpose of `conan config install` is to setup everything in user's conan home automatically in one shot.

`wget` has `--no-check-certificate` that is explicit, `curl` has `--insecure` that is a bit obscure from my point of view, I would propose `--ssl-verify=False` so that the relation with ssl is clear.

Thanks for your help and reactivity.
For me the `--ssl-verify=False` as a parameter of the `conan config install` is ok. I agree that it is a very common argument in some client tools. 
Ok, then. Lets do ``ssl-verify``.
I have assigned it to @danimtb for 1.2, but it shouldn't be difficult to implement, so if possible to speed it up to 1.1, we might try.

Resolving patch:
diff --git a/conans/client/command.py b/conans/client/command.py
--- a/conans/client/command.py
+++ b/conans/client/command.py
@@ -290,6 +290,10 @@ def config(self, *args):
         get_subparser.add_argument("item", nargs="?", help="item to print")
         set_subparser.add_argument("item", help="key=value to set")
         install_subparser.add_argument("item", nargs="?", help="configuration file to use")
+
+        install_subparser.add_argument("--verify-ssl", nargs="?", default="True",
+                                       help='Verify SSL connection when downloading file')
+
         args = parser.parse_args(*args)
 
         if args.subcommand == "set":
@@ -303,7 +307,8 @@ def config(self, *args):
         elif args.subcommand == "rm":
             return self._conan.config_rm(args.item)
         elif args.subcommand == "install":
-            return self._conan.config_install(args.item)
+            verify_ssl = get_bool_from_text(args.verify_ssl)
+            return self._conan.config_install(args.item, verify_ssl)
 
     def info(self, *args):
         """Gets information about the dependency graph of a recipe.
diff --git a/conans/client/conan_api.py b/conans/client/conan_api.py
--- a/conans/client/conan_api.py
+++ b/conans/client/conan_api.py
@@ -437,9 +437,9 @@ def config_rm(self, item):
         self._client_cache.invalidate()
 
     @api_method
-    def config_install(self, item):
+    def config_install(self, item, verify_ssl):
         from conans.client.conf.config_installer import configuration_install
-        return configuration_install(item, self._client_cache, self._user_io.out, self._runner)
+        return configuration_install(item, self._client_cache, self._user_io.out, self._runner, verify_ssl)
 
     def _info_get_profile(self, reference, install_folder, profile_name, settings, options, env):
         cwd = os.getcwd()
diff --git a/conans/client/conf/config_installer.py b/conans/client/conf/config_installer.py
--- a/conans/client/conf/config_installer.py
+++ b/conans/client/conf/config_installer.py
@@ -38,11 +38,11 @@ def _handle_profiles(source_folder, target_folder, output):
             shutil.copy(os.path.join(root, f), os.path.join(target_folder, profile))
 
 
-def _process_git_repo(repo_url, client_cache, output, runner, tmp_folder):
+def _process_git_repo(repo_url, client_cache, output, runner, tmp_folder, verify_ssl):
     output.info("Trying to clone repo  %s" % repo_url)
 
     with tools.chdir(tmp_folder):
-        runner('git clone "%s" config' % repo_url, output=output)
+        runner('git -c http.sslVerify=%s clone "%s" config' % (verify_ssl, repo_url), output=output)
     tmp_folder = os.path.join(tmp_folder, "config")
     _process_folder(tmp_folder, client_cache, output)
 
@@ -90,14 +90,14 @@ def _process_folder(folder, client_cache, output):
         dirs[:] = [d for d in dirs if d not in ("profiles", ".git")]
 
 
-def _process_download(item, client_cache, output, tmp_folder):
+def _process_download(item, client_cache, output, tmp_folder, verify_ssl):
     output.info("Trying to download  %s" % _hide_password(item))
     zippath = os.path.join(tmp_folder, "config.zip")
-    tools.download(item, zippath, out=output)
+    tools.download(item, zippath, out=output, verify=verify_ssl)
     _process_zip_file(zippath, client_cache, output, tmp_folder, remove=True)
 
 
-def configuration_install(item, client_cache, output, runner):
+def configuration_install(item, client_cache, output, runner, verify_ssl):
     tmp_folder = os.path.join(client_cache.conan_folder, "tmp_config_install")
     # necessary for Mac OSX, where the temp folders in /var/ are symlinks to /private/var/
     tmp_folder = os.path.realpath(tmp_folder)
@@ -111,12 +111,12 @@ def configuration_install(item, client_cache, output, runner):
                                      "'general.config_install' not defined in conan.conf")
 
         if item.endswith(".git"):
-            _process_git_repo(item, client_cache, output, runner, tmp_folder)
+            _process_git_repo(item, client_cache, output, runner, tmp_folder, verify_ssl)
         elif os.path.exists(item):
             # is a local file
             _process_zip_file(item, client_cache, output, tmp_folder)
         elif item.startswith("http"):
-            _process_download(item, client_cache, output, tmp_folder)
+            _process_download(item, client_cache, output, tmp_folder, verify_ssl)
         else:
             raise ConanException("I don't know how to process %s" % item)
     finally:


Test patch:
diff --git a/conans/test/command/config_install_test.py b/conans/test/command/config_install_test.py
--- a/conans/test/command/config_install_test.py
+++ b/conans/test/command/config_install_test.py
@@ -266,3 +266,20 @@ def my_download(obj, url, filename, **kwargs):  # @UnusedVariable
 
             # Check credentials still stored in configuration
             self._check(fake_url_with_credentials)
+
+    def ssl_verify_test(self):
+        fake_url = "https://fakeurl.com/myconf.zip"
+
+        def download_verify_false(obj, url, filename, **kwargs):  # @UnusedVariable
+            self.assertFalse(obj.verify)
+            self._create_zip(filename)
+
+        def download_verify_true(obj, url, filename, **kwargs):  # @UnusedVariable
+            self.assertTrue(obj.verify.endswith("cacert.pem"))
+            self._create_zip(filename)
+
+        with patch.object(Downloader, 'download', new=download_verify_false):
+            self.client.run("config install %s --verify-ssl=False" % fake_url)
+
+        with patch.object(Downloader, 'download', new=download_verify_true):
+            self.client.run("config install %s --verify-ssl=True" % fake_url)