Repository: ansible/ansible

Issue:
WinRM Connectionplugin falls back to defaults when an ignored templating error  exists in hostvars
##### ISSUE TYPE
 - Bug Report

##### COMPONENT NAME
module/plugin/connection/winrm.py


##### ANSIBLE VERSION
```
ansible 2.4.0.0
  config file = /opt/it-infra/ansible.cfg
  configured module search path = [u'/opt/it-infra/library']
  ansible python module location = /home/vagrant/.local/lib/python2.7/site-packages/ansible
  executable location = /usr/local/bin/ansible
  python version = 2.7.12 (default, Nov 19 2016, 06:48:10) [GCC 5.4.0 20160609]
```

##### CONFIGURATION
```
ANSIBLE_NOCOWS(/opt/it-infra/ansible.cfg) = False
DEFAULT_CALLBACK_WHITELIST(/opt/it-infra/ansible.cfg) = ['profile_tasks', 'timer']
DEFAULT_FILTER_PLUGIN_PATH(/opt/it-infra/ansible.cfg) = [u'/opt/it-infra/filter_plugins']
DEFAULT_MODULE_PATH(/opt/it-infra/ansible.cfg) = [u'/opt/it-infra/library']
DEFAULT_ROLES_PATH(/opt/it-infra/ansible.cfg) = [u'/opt/it-infra/roles']
DEFAULT_UNDEFINED_VAR_BEHAVIOR(/opt/it-infra/ansible.cfg) = True
DEFAULT_VAULT_PASSWORD_FILE(/opt/it-infra/ansible.cfg) = /opt/it-infra/vpass.py
DISPLAY_SKIPPED_HOSTS(/opt/it-infra/ansible.cfg) = False
HOST_KEY_CHECKING(/opt/it-infra/ansible.cfg) = False
RETRY_FILES_ENABLED(/opt/it-infra/ansible.cfg) = False
```


##### OS / ENVIRONMENT
Host: Ubuntu 16.04.2
Target: Server 2016

##### SUMMARY
When a host_var file exists with content of

```yaml
exchange:
  mailbox:
    employe_database_defaults:
      # Defined in Seconds maybe 7776000 = 90 Tage
      deleted_item_retention: 7776000
      prohibit_send_quota: 0
      prohibit_send_receive_quota: 0
      retain_deleted_items_until_backup: true

    database:
      # Employe Mailbox Database
      - name: "fj-v-mx1-e-db1"
        size: 200000
        quota: 2000
        path_db: 'D:/database/fj-v-mx1-e-db1.edb'
        path_log_dir: 'D:/log/fj-v-mx1-e-db1'
        deleted_item_retention: "{{ exchange.mailbox.database.employe_database_defaults.deleted_item_retention }}"
```

**winrm.py has an empty hostvars object, so all connection settings are missing on processing**

##### STEPS TO REPRODUCE

host_var/fj-v-mx1/exchange.yml
```yaml
exchange:
  mailbox:
    employe_database_defaults:
      # Defined in Seconds maybe 7776000 = 90 Tage
      deleted_item_retention: 7776000
      prohibit_send_quota: 0
      prohibit_send_receive_quota: 0
      retain_deleted_items_until_backup: true

    database:
      # Employe Mailbox Database
      - name: "fj-v-mx1-e-db1"
        size: 200000
        quota: 2000
        path_db: 'D:/database/fj-v-mx1-e-db1.edb'
        path_log_dir: 'D:/log/fj-v-mx1-e-db1'
        deleted_item_retention: "{{ exchange.mailbox.database.employe_database_defaults.deleted_item_retention }}"
```

hostfile simple.ini:
```ini
fj-v-mx1 ansible_host=172.16.21.50

[testing]
fj-v-mx1

[testing:vars]
ansible_user="ansible"
ansible_password="xxxx"
ansible_port=5986
ansible_connection=winrm
ansible_winrm_server_cert_validation=ignore
ansible_winrm_scheme=https
ansible_winrm_transport=ssl
```

Command:
```bash
ansible -m win_ping fj-v-mx1 -i simple.ini
```

<!--- You can also paste gist.github.com links for larger files -->

##### EXPECTED RESULTS
win_ping module success

##### ACTUAL RESULTS
SSL Certificate Error of WinRM Plugin, because winrm.py got an empty hostvars object and
so missing the 'ansible_winrm_server_cert_validation' option

<!--- Paste verbatim command output between quotes below -->
```
Using module file /usr/local/lib/python2.7/dist-packages/ansible/modules/windows/setup.ps1
<172.16.21.50> ESTABLISH WINRM CONNECTION FOR USER: ansible@JOSEPHINUM.AT on PORT 5986 TO 172.16.21.50
creating Kerberos CC at /tmp/tmp5iFT_j
calling kinit for principal ansible@JOSEPHINUM.AT
kinit succeeded for principal ansible@JOSEPHINUM.AT
<172.16.21.50> WINRM CONNECT: transport=kerberos endpoint=https://172.16.21.50:5986/wsman
<172.16.21.50> WINRM CONNECTION ERROR: [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed (_ssl.c:590)
Traceback (most recent call last):
  File "/usr/local/lib/python2.7/dist-packages/ansible/plugins/connection/winrm.py", line 219, in _winrm_connect
    self.shell_id = protocol.open_shell(codepage=65001)  # UTF-8
  File "/home/vagrant/.local/lib/python2.7/site-packages/winrm/protocol.py", line 132, in open_shell
    res = self.send_message(xmltodict.unparse(req))
  File "/home/vagrant/.local/lib/python2.7/site-packages/winrm/protocol.py", line 207, in send_message
    return self.transport.send_message(message)
  File "/home/vagrant/.local/lib/python2.7/site-packages/winrm/transport.py", line 184, in send_message
    response = self.session.send(prepared_request, timeout=self.read_timeout_sec)
  File "/home/vagrant/.local/lib/python2.7/site-packages/requests/sessions.py", line 642, in send
    r = adapter.send(request, **kwargs)
  File "/home/vagrant/.local/lib/python2.7/site-packages/requests/adapters.py", line 513, in send
    raise SSLError(e, request=request)
SSLError: [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed (_ssl.c:590)

<172.16.21.50> WINRM CONNECT: transport=ssl endpoint=https://172.16.21.50:5986/wsman
<172.16.21.50> WINRM CONNECTION ERROR: [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed (_ssl.c:590)
Traceback (most recent call last):
  File "/usr/local/lib/python2.7/dist-packages/ansible/plugins/connection/winrm.py", line 219, in _winrm_connect
    self.shell_id = protocol.open_shell(codepage=65001)  # UTF-8
  File "/home/vagrant/.local/lib/python2.7/site-packages/winrm/protocol.py", line 132, in open_shell
    res = self.send_message(xmltodict.unparse(req))
  File "/home/vagrant/.local/lib/python2.7/site-packages/winrm/protocol.py", line 207, in send_message
    return self.transport.send_message(message)
  File "/home/vagrant/.local/lib/python2.7/site-packages/winrm/transport.py", line 184, in send_message
    response = self.session.send(prepared_request, timeout=self.read_timeout_sec)
  File "/home/vagrant/.local/lib/python2.7/site-packages/requests/sessions.py", line 642, in send
    r = adapter.send(request, **kwargs)
  File "/home/vagrant/.local/lib/python2.7/site-packages/requests/adapters.py", line 513, in send
    raise SSLError(e, request=request)
SSLError: [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed (_ssl.c:590)

fatal: [fj-v-mx1]: UNREACHABLE! => {
    "changed": false,
    "msg": "kerberos: [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed (_ssl.c:590), ssl: [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed (_ssl.c:590)",
    "unreachable": true
}
```

---

Looks like when i put the Defaults from a other defined var into it , see host_vars file, all hostvar content in winrm.py moduel getting cleared.

Remove follow line in host_var file:
```yaml
 deleted_item_retention: "{{ exchange.mailbox.database.employe_database_defaults.deleted_item_retention }}"
```` 
WinRM Connection plugin don't complain about Certificate , because the hostvar object is not empty

Could not find where in the upper execution chain, somewhere on variable file parsing the problem exist


Discussion:

Resolving patch:
diff --git a/lib/ansible/executor/task_executor.py b/lib/ansible/executor/task_executor.py
--- a/lib/ansible/executor/task_executor.py
+++ b/lib/ansible/executor/task_executor.py
@@ -474,18 +474,11 @@ def _execute(self, variables=None):
                 not getattr(self._connection, 'connected', False) or
                 self._play_context.remote_addr != self._connection._play_context.remote_addr):
             self._connection = self._get_connection(variables=variables, templar=templar)
-            hostvars = variables.get('hostvars', None)
             # only template the vars if the connection actually implements set_host_overrides
             # NB: this is expensive, and should be removed once connection-specific vars are being handled by play_context
             sho_impl = getattr(type(self._connection), 'set_host_overrides', None)
-            if hostvars and sho_impl and sho_impl != ConnectionBase.set_host_overrides:
-                try:
-                    target_hostvars = hostvars.get(self._host.name)
-                except:
-                    # FIXME: this should catch the j2undefined error here
-                    #        specifically instead of all exceptions
-                    target_hostvars = dict()
-                self._connection.set_host_overrides(host=self._host, hostvars=target_hostvars)
+            if sho_impl and sho_impl != ConnectionBase.set_host_overrides:
+                self._connection.set_host_overrides(self._host, variables, templar)
         else:
             # if connection is reused, its _play_context is no longer valid and needs
             # to be replaced with the one templated above, in case other data changed
diff --git a/lib/ansible/plugins/connection/winrm.py b/lib/ansible/plugins/connection/winrm.py
--- a/lib/ansible/plugins/connection/winrm.py
+++ b/lib/ansible/plugins/connection/winrm.py
@@ -33,12 +33,10 @@
 import os
 import re
 import shlex
-import socket
 import traceback
 import json
 import tempfile
 import subprocess
-import itertools
 
 HAVE_KERBEROS = False
 try:
@@ -54,7 +52,7 @@
 from ansible.module_utils._text import to_bytes, to_native, to_text
 from ansible.module_utils.six import binary_type
 from ansible.plugins.connection import ConnectionBase
-from ansible.plugins.shell.powershell import exec_wrapper, become_wrapper, leaf_exec
+from ansible.plugins.shell.powershell import leaf_exec
 from ansible.utils.hashing import secure_hash
 from ansible.utils.path import makedirs_safe
 
@@ -100,13 +98,18 @@ def __init__(self, *args, **kwargs):
 
         super(Connection, self).__init__(*args, **kwargs)
 
-    def set_host_overrides(self, host, hostvars=None):
+    def set_host_overrides(self, host, variables, templar):
         '''
         Override WinRM-specific options from host variables.
         '''
         if not HAS_WINRM:
             return
 
+        hostvars = {}
+        for k in variables:
+            if k.startswith('ansible_winrm'):
+                hostvars[k] = templar.template(variables[k])
+
         self._winrm_host = self._play_context.remote_addr
         self._winrm_port = int(self._play_context.port or 5986)
         self._winrm_scheme = hostvars.get('ansible_winrm_scheme', 'http' if self._winrm_port == 5985 else 'https')


Test patch: