Repository: ansible/ansible

Issue:
ssl support for lookup consul_kv
##### SUMMARY
The lookup plugin consul_kv does not support to change the scheme to https not to send a cert with the request nor to skip cert verifiaction if that is desired

##### ISSUE TYPE
 - Feature Idea

##### COMPONENT NAME
lookup/consul_kv

##### ANSIBLE VERSION
```
ansible 2.5.3 (stable-2.5 9d8d1de182) last updated 2018/05/22 09:25:46 (GMT +200)
  config file = /mnt/c/Projekte/ansible/ansible.cfg
  configured module search path = [u'/root/.ansible/plugins/modules', u'/usr/share/ansible/plugins/modules']
  ansible python module location = /opt/ansible/lib/ansible
  executable location = /opt/ansible/bin/ansible
  python version = 2.7.12 (default, Dec  4 2017, 14:50:18) [GCC 5.4.0 20160609]
```

##### CONFIGURATION
ANSIBLE_PIPELINING(/mnt/c/Projekte/ansible/ansible.cfg) = True
ANSIBLE_SSH_ARGS(/mnt/c/Projekte/ansible/ansible.cfg) = -C -o ControlMaster=auto -o ControlPersist=60s -o UserKnownHostsFile=/dev/null
DEFAULT_CALLBACK_WHITELIST(/mnt/c/Projekte/ansible/ansible.cfg) = ['timer', 'mail']
DEFAULT_FORKS(/mnt/c/Projekte/ansible/ansible.cfg) = 20
DEFAULT_GATHERING(/mnt/c/Projekte/ansible/ansible.cfg) = smart
DEFAULT_LOG_PATH(/mnt/c/Projekte/ansible/ansible.cfg) = /var/log/ansible.log
DEFAULT_PRIVATE_KEY_FILE(/mnt/c/Projekte/ansible/ansible.cfg) = /var/lib/rundeck/.ssh/ec2_user_id_rsa
DEFAULT_REMOTE_USER(/mnt/c/Projekte/ansible/ansible.cfg) = ec2-user
DEFAULT_SCP_IF_SSH(/mnt/c/Projekte/ansible/ansible.cfg) = smart
DISPLAY_SKIPPED_HOSTS(/mnt/c/Projekte/ansible/ansible.cfg) = True
HOST_KEY_CHECKING(/mnt/c/Projekte/ansible/ansible.cfg) = False
RETRY_FILES_ENABLED(/mnt/c/Projekte/ansible/ansible.cfg) = False

##### OS / ENVIRONMENT
N/A

##### STEPS TO REPRODUCE
```yaml
 tasks:
    - name: "check for consul entry for module {{ item.name }}"
      set_fact:
        consul_entry: "{{ lookup('consul_kv',
          'my/key/path',
          host='myConsulHost', scheme='https', port=443, verify=False) }}"
```
##### EXPECTED RESULTS
The key value returned

##### ACTUAL RESULTS
```
TASK [check for consul entry for module someModule] *****************************************************************************************************************************************************************************************
task path: /mnt/c/Projekte/ansible/test_johannes.yml:14
fatal: [localhost]: FAILED! => {
    "msg": "An unhandled exception occurred while running the lookup plugin 'consul_kv'. Error was a <class 'ansible.errors.AnsibleError'>, original message: Error locating 'my/key/path' in kv store. Error was ('Connection aborted.', BadStatusLine('\\x15\\x03\\x01\\x00\\x02\\x02\\n',))"
}
```



Discussion:
Files identified in the description:
* [lib/ansible/modules/clustering/consul_kv.py](https://github.com/ansible/ansible/blob/devel/lib/ansible/modules/clustering/consul_kv.py)

If these files are inaccurate, please update the `component name` section of the description or use the `!component` bot command.

[click here for bot help](https://github.com/ansible/ansibullbot/blob/master/ISSUE_HELP.md)
<!--- boilerplate: components_banner --->
cc @colin-nolan @sgargan
[click here for bot help](https://github.com/ansible/ansibullbot/blob/master/ISSUE_HELP.md)
<!--- boilerplate: notify --->
Hm... actually this affects the lookup plugin, not the clustering plugin
Files identified in the description:
* [lib/ansible/plugins/lookup/consul_kv.py](https://github.com/ansible/ansible/blob/devel/lib/ansible/plugins/lookup/consul_kv.py)

If these files are inaccurate, please update the `component name` section of the description or use the `!component` bot command.

[click here for bot help](https://github.com/ansible/ansibullbot/blob/master/ISSUE_HELP.md)
<!--- boilerplate: components_banner --->

Resolving patch:
diff --git a/lib/ansible/plugins/lookup/consul_kv.py b/lib/ansible/plugins/lookup/consul_kv.py
--- a/lib/ansible/plugins/lookup/consul_kv.py
+++ b/lib/ansible/plugins/lookup/consul_kv.py
@@ -2,12 +2,13 @@
 # (c) 2017 Ansible Project
 # GNU General Public License v3.0+ (see COPYING or https://www.gnu.org/licenses/gpl-3.0.txt)
 from __future__ import (absolute_import, division, print_function)
+
 __metaclass__ = type
 
 DOCUMENTATION = """
     lookup: consul_kv
     version_added: "1.9"
-    short_description: Fetch  metadata from a Consul key value store.
+    short_description: Fetch metadata from a Consul key value store.
     description:
       - Lookup metadata for a playbook from the key value store in a Consul cluster.
         Values can be easily set in the kv store with simple rest commands
@@ -24,18 +25,51 @@
         description: If true, will retrieve all the values that have the given key as prefix.
         default: False
       index:
-        description: If the key has a value with the specified index then this is returned allowing access to historical values.
+        description:
+          - If the key has a value with the specified index then this is returned allowing access to historical values.
       token:
         description: The acl token to allow access to restricted values.
       host:
         default: localhost
         description:
-           - The target to connect to, must be a resolvable address.
+          - The target to connect to, must be a resolvable address.
+            Will be determined from C(ANSIBLE_CONSUL_URL) if that is set.
+          - "C(ANSIBLE_CONSUL_URL) should look like this: C(https://my.consul.server:8500)"
         env:
           - name: ANSIBLE_CONSUL_URL
+        ini:
+          - section: lookup_consul
+            key: host
+        version_added: "2.8"
       port:
-        description: The port of the target host to connect to.
+        description:
+          - The port of the target host to connect to.
+          - If you use C(ANSIBLE_CONSUL_URL) this value will be used from there.
         default: 8500
+      scheme:
+        default: http
+        description:
+          - Whether to use http or https.
+          - If you use C(ANSIBLE_CONSUL_URL) this value will be used from there.
+        version_added: "2.8"
+      validate_certs:
+        default: True
+        description: Whether to verify the ssl connection or not.
+        env:
+          - name: ANSIBLE_CONSUL_VALIDATE_CERTS
+        ini:
+          - section: lookup_consul
+            key: validate_certs
+        version_added: "2.8"
+      client_cert:
+        default: None
+        description: The client cert to verify the ssl connection.
+        env:
+          - name: ANSIBLE_CONSUL_CLIENT_CERT
+        ini:
+          - section: lookup_consul
+            key: client_cert
+        version_added: "2.8"
 """
 
 EXAMPLES = """
@@ -62,7 +96,6 @@
 """
 
 import os
-import sys
 from ansible.module_utils.six.moves.urllib.parse import urlparse
 from ansible.errors import AnsibleError, AnsibleAssertionError
 from ansible.plugins.lookup import LookupBase
@@ -74,6 +107,7 @@
 
 try:
     import consul
+
     HAS_CONSUL = True
 except ImportError as e:
     HAS_CONSUL = False
@@ -84,7 +118,8 @@ class LookupModule(LookupBase):
     def run(self, terms, variables=None, **kwargs):
 
         if not HAS_CONSUL:
-            raise AnsibleError('python-consul is required for consul_kv lookup. see https://python-consul.readthedocs.io/en/latest/#installation')
+            raise AnsibleError(
+                'python-consul is required for consul_kv lookup. see http://python-consul.readthedocs.org/en/latest/#installation')
 
         values = []
         try:
@@ -92,12 +127,19 @@ def run(self, terms, variables=None, **kwargs):
                 params = self.parse_params(term)
                 try:
                     url = os.environ['ANSIBLE_CONSUL_URL']
+                    validate_certs = os.environ['ANSIBLE_CONSUL_VALIDATE_CERTS'] or True
+                    client_cert = os.environ['ANSIBLE_CONSUL_CLIENT_CERT'] or None
                     u = urlparse(url)
-                    consul_api = consul.Consul(host=u.hostname, port=u.port, scheme=u.scheme)
+                    consul_api = consul.Consul(host=u.hostname, port=u.port, scheme=u.scheme, verify=validate_certs,
+                                               cert=client_cert)
                 except KeyError:
                     port = kwargs.get('port', '8500')
                     host = kwargs.get('host', 'localhost')
-                    consul_api = consul.Consul(host=host, port=port)
+                    scheme = kwargs.get('scheme', 'http')
+                    validate_certs = kwargs.get('validate_certs', True)
+                    client_cert = kwargs.get('client_cert', None)
+                    consul_api = consul.Consul(host=host, port=port, scheme=scheme, verify=validate_certs,
+                                               cert=client_cert)
 
                 results = consul_api.kv.get(params['key'],
                                             token=params['token'],


Test patch: