Repository: ansible/ansible

Issue:
template generates strange error message when it has non ascii charcters, even in comments
##### ISSUE TYPE
- Bug Report
##### ANSIBLE VERSION

```
ansible 2.0.1.0
```
##### OS / ENVIRONMENT

Debian Jessie amd64

Writing {% elseif ... %} instead of {% elif ... %} in a Jinja2 template generates strange error messages for a template that contains UTF-8 characters:

```
 [WARNING]: Failure when attempting to use callback plugin (<ansible.plugins.callback.default.CallbackModule object at 0x7fc60880f8d0>): 'ascii' codec can't decode byte 0xc4
in position 2242: ordinal not in range(128)
```

I thought that there was somethins wrong with the file encoding, when in fact the problem was caused by writing "elseif" instead of "elif" in a conditional statement.



Discussion:
I cannot reproduce this:

**Ansible v1.9.5** --> Error printed twice !

```
fatal: [localhost] => {'msg': "AnsibleError: file: /home/dag/home-made/ansible.testing/files/test91.j2, line number: 3, error: unknown tag 'elseif'", 'failed': True}
fatal: [localhost] => {'msg': "AnsibleError: file: /home/dag/home-made/ansible.testing/files/test91.j2, line number: 3, error: unknown tag 'elseif'", 'failed': True}
```

**Ansible v2.0.1.0**

```
fatal: [localhost]: FAILED! => {"changed": false, "failed": true, "msg": "AnsibleError: template error while templating string: unknown tag 'elseif'. String: {% if inventory_hostname != 'localhost' %}\nSOME OTHER HOST\n{% elseif %}\nLOCALHOST\n{% endif %}"}
```

**Ansible v2.0.2.0**

```
fatal: [localhost]: FAILED! => {"changed": false, "failed": true, "msg": "AnsibleError: template error while templating string: unknown tag 'elseif'. String: {% if inventory_hostname != 'localhost' %}\nSOME OTHER HOST\n{% elseif %}\nLOCALHOST\n{% endif %}"}
```

The error seems fine to me.
Can you please investigate to see what is special in your environment that could trigger a difference ? Maybe also share a minimal reproducing template file so we can see if we can reproduce the exact error ?

template.txt

```
# treść

{% if zzz is defined %}

xxx

{% elseif xxx is defined %}

yyy

{% endif %}
```

playbook.yml

```
---
- hosts: server-1000.example.com
  tasks:
    - name: Test
      template: src=template.txt dest="/tmp/template-output.txt" owner=root group=root mode=0640
      delegate_to: local.example.com
```

Note that I delegate the task to a local server where Ansible is running (local.example.com). The error message I get:

```
 [WARNING]: Failure when attempting to use callback plugin (<ansible.plugins.callback.default.CallbackModule object at 0x7fba04ccc990>): 'ascii' codec can't decode byte 0xc5
in position 269: ordinal not in range(128)
```

When I fix the template (replacing "elseif" with "elif") everything works as expected. But it shows that there is something wrong with the way Ansible / Jinja2 parses templates with UTF-8 characters and it makes debugging errors in such templates much harder.

what happens if you remove the comment? `# treść`

Then I get a correct error message:

```
fatal: [server-1000.example.com -> local.example.com]: FAILED! => {"changed": false, "failed": true, "msg": "AnsibleError: template error while templating string: Encountered unknown tag 'elseif'. Jinja was looking for the following tags: 'elif' or 'else' or 'endif'. The innermost block that needs to be closed is 'if'.. String: \n{% if zzz is defined %}\n\nxxx\n\n{% elseif xxx is defined %}\n\nyyy\n\n{% endif %}\n"}
```

It seems strange that UTF-8 characters (in this case "ć") change the way in which Jinja2 parses the template.

@pako-pl Yes, you found a genuine bug. And a very strange at that.

Okay, this is just about the error message we get.  Looking into the callback plugin to figure out what's wrong there.


Resolving patch:
diff --git a/lib/ansible/executor/task_executor.py b/lib/ansible/executor/task_executor.py
--- a/lib/ansible/executor/task_executor.py
+++ b/lib/ansible/executor/task_executor.py
@@ -26,7 +26,7 @@
 import time
 import traceback
 
-from ansible.compat.six import iteritems, string_types
+from ansible.compat.six import iteritems, string_types, binary_type
 
 from ansible import constants as C
 from ansible.errors import AnsibleError, AnsibleParserError, AnsibleUndefinedVariable, AnsibleConnectionFailure
@@ -137,6 +137,8 @@ def _clean_res(res):
                         res[idx] = _clean_res(item)
                 elif isinstance(res, UnsafeProxy):
                     return res._obj
+                elif isinstance(res, binary_type):
+                    return to_unicode(res, errors='strict')
                 return res
 
             display.debug("dumping result to json")


Test patch: