Repository: ansible/ansible

Issue:
safe_eval prevents lookups from returning JSON
`safe_eval,` due to the fact that it uses ast and eval, prevents a lookup from returning JSON or a string that evals to a python data structure other than a string.

Take the example of:

```
lookup('template', 'build_json.json.j2')
```

`safe_json` will turn that output into a dict/list, instead of allowing JSON.

Before dc48d27dd2c49e95a719d7f7596f0b68e0ddef7f this worked ok, but largely due to a bug, that caused a `NameError` for `true`, `false` and `null`, as they could not be evaled.  As a result, the non-evaled string would be returned, instead of an evaluated string.

Now, because those values are converted, the lookup returns a dict.

This is a further problem, as `Templar._lookup` will raise `TypeError: sequence item 0: expected string, dict found`.

This represents as the following error message:

```
fatal: [foo]: FAILED! => {"failed": true, "msg": "an unexpected type error occurred. Error was sequence item 0: expected string, dict found"}
```

Reproducer:
##### playbook

```

---
- hosts: all
  gather_facts: false
  tasks:
    - debug: msg="{{ lookup('template', 'json.j2') }}"
```
##### template

```
{"hostname": "{{ inventory_hostname }}"}
```



Discussion:
you should be able to bypass it by using the `|to_json` filter.

An exception is raised in `_lookup` before the data can even make it to the `to_json` filter when the data returned from the lookup plugin is passed through `join`

I wonder if we can just drop the join, and always return a list?  The only side effect, is that with the join, a list with only a single item, turns out to only be that single item, instead of a list with a single element.

Maybe we could return a list if many, or just the element and no list if 1, and enable the `with_` loops to detect that and properly loop over a non-list item, by making it a list.

Actually, that may already be possible, as the following change seems to work fairly well in my simple tests so far:

``` diff
diff --git a/lib/ansible/template/__init__.py b/lib/ansible/template/__init__.py
index bec34f8..ae98245 100644
--- a/lib/ansible/template/__init__.py
+++ b/lib/ansible/template/__init__.py
@@ -401,8 +401,6 @@ class Templar:
         instance = self._lookup_loader.get(name.lower(), loader=self._loader, templar=self)

         if instance is not None:
-            wantlist = kwargs.pop('wantlist', False)
-
             from ansible.utils.listify import listify_lookup_plugin_terms
             loop_terms = listify_lookup_plugin_terms(terms=args, templar=self, loader=self._loader, fail_on_undefined=True, convert_bare=False)
             # safely catch run failures per #5059
@@ -417,10 +415,10 @@ class Templar:

             if ran:
                 from ansible.vars.unsafe_proxy import UnsafeProxy, wrap_var
-                if wantlist:
-                    ran = wrap_var(ran)
+                if len(ran) == 1:
+                    return wrap_var(ran[0])
                 else:
-                    ran = UnsafeProxy(",".join(ran))
+                    return wrap_var(ran)

             return ran
         else:
```

This also allows us to drop `wantlist` as it will always return a list instead of a `,` joined string.

It doesn't change the fact that the JSON string is turned into a dict, but since not turning it into a dict only happened when `true`, `false`, or `null` was present, that would at least leave thing consistent.


Resolving patch:
diff --git a/lib/ansible/template/__init__.py b/lib/ansible/template/__init__.py
--- a/lib/ansible/template/__init__.py
+++ b/lib/ansible/template/__init__.py
@@ -420,7 +420,13 @@ def _lookup(self, name, *args, **kwargs):
                 if wantlist:
                     ran = wrap_var(ran)
                 else:
-                    ran = UnsafeProxy(",".join(ran))
+                    try:
+                        ran = UnsafeProxy(",".join(ran))
+                    except TypeError:
+                        if isinstance(ran, list) and len(ran) == 1:
+                            ran = wrap_var(ran[0])
+                        else:
+                            ran = wrap_var(ran)
 
             return ran
         else:


Test patch: