Repository: ansible/ansible

Issue:
fact caching breaks ansible_facts access when an env var has {%%}
##### SUMMARY
On a host, I use zsh (oh-my-zsh if that makes a difference) with a fairly customized prompt. I have the env var:
```
RPROMPT=$(vi_mode_prompt_info)%{%K{default}%}%{%F{yellow}%} %L
```

##### ISSUE TYPE
 - Bug Report

##### COMPONENT NAME
- jsonfile cache
- env var fact gatherer

##### ANSIBLE VERSION
```
ansible 2.6.1
  config file = /etc/ansible/ansible.cfg
  configured module search path = [u'/home/cognifloyd/.ansible/plugins/modules', u'/usr/share/ansible/plugins/modules']
  ansible python module location = /usr/lib/python2.7/site-packages/ansible
  executable location = /usr/bin/ansible
  python version = 2.7.5 (default, Nov  6 2016, 00:28:07) [GCC 4.8.5 20150623 (Red Hat 4.8.5-11)]
```

##### CONFIGURATION
```
ANSIBLE_SSH_CONTROL_PATH(/etc/ansible/ansible.cfg) = %(directory)s/%%h-%%p-%%r
ANSIBLE_SSH_CONTROL_PATH_DIR(/etc/ansible/ansible.cfg) = ~/.tmp/ssh-cp
CACHE_PLUGIN(/etc/ansible/ansible.cfg) = jsonfile
CACHE_PLUGIN_CONNECTION(/etc/ansible/ansible.cfg) = /var/tmp/ansible_fact_cache
CACHE_PLUGIN_TIMEOUT(/etc/ansible/ansible.cfg) = 86400
DEFAULT_FORKS(/etc/ansible/ansible.cfg) = 20
DEFAULT_GATHERING(/etc/ansible/ansible.cfg) = smart

DEFAULT_..._PLUGIN/MODULE/ROLE_PATH... = [customized to add in ~]
DEFAULT_TIMEOUT(/etc/ansible/ansible.cfg) = 30
DEFAULT_VAULT_PASSWORD_FILE(/etc/ansible/ansible.cfg) = /some/file/here
HOST_KEY_CHECKING(/etc/ansible/ansible.cfg) = True
INVENTORY_ENABLED(/etc/ansible/ansible.cfg) = ['host_list', 'mycustom', 'yaml', 'constructed']
```

##### OS / ENVIRONMENT
From CentOS to CentOS (controller and managed host are the same machine in this test).
OS shouldn't matter.

##### STEPS TO REPRODUCE
0. Add a var with a value like {%K%} to your bashrc or zshrc file so that it gets loaded gathering facts.
1. enable fact cache and run setup
2. try to access ansible_facts.distribution

```sh
$ export ANSIBLE_CACHE_PLUGIN=jsonfile
$ export ANSIBLE_CACHE_PLUGIN_CONNECTION=/tmp/ansible_facts
$ ansible -m setup localhost
$ ansible -m debug -a var=ansible_facts.distribution localhost
```

##### EXPECTED RESULTS
Accessing ansible_facts works.

##### ACTUAL RESULTS
Crazy error that spits out all of the ansible_facts as a big long string.

snipped output below
```
localhost | FAILED! => {
    "msg": "An unhandled exception occurred while templating '{u'lsb': {}, u'selinux': {u'status': u'disabled'}, u'module_setup': True, ... u'env': {u'HISTTIMEFORMAT': u'%m/%d/%y %T ', u'LC_NUMERIC': u'en_US.UTF-8', ... u'LOGNAME': u'jacob', u'USER': u'jacob', u'PATH': u'/usr/local/bin:/usr/bin:/home/jacob/bin:/usr/local/sbin:/usr/sbin:/home/jacob/.local/bin', u'HOME': u'/home/jacob', u'ZSH': u'/home/jacob/.oh-my-zsh', u'LANG': u'en_US.UTF-8', u'TERM': u'xterm-256color', u'RPROMPT': u'$(vi_mode_prompt_info)%{%K{default}%}%{%F{yellow}%} %L', ...}, ...}'. Error was a <class 'ansible.errors.AnsibleError'>, original message: template error while templating string: Encountered unknown tag 'K'.. String: $(vi_mode_prompt_info)%{%K{default}%}%{%F{yellow}%} %L"
```



Discussion:
@agaffney You said to tag you.
```
(22:13:56) agaffney: yeah, my error is different, but for a similar reason
(22:14:07) agaffney: mine is happening because of the way I'm trying to reproduce it
(22:14:18) agaffney: I'd need to actually modify my host env to reproduce
(22:16:17) agaffney: okay, now I can't reproduce it
(22:17:49) agaffney: I can't trigger it with 'debug: var=ansible_facts.distribution' or 'when: ansible_facts.distribution is not defined', but I see the "bad" string in ansible_facts.env
(22:18:02) agaffney: I tested with 2.6.1 and devel
(22:18:23) cognifloyd: I'm in 2.5.5
(22:19:15) cognifloyd: I wonder if it is only triggered because I have a fact cache?
(22:19:18) agaffney: I can't reproduce with the v2.5.5 tag in git, either
(22:19:23) agaffney: possibly
(22:19:52) cognifloyd: I edited the cache (jsonfile), deleting the RPROMPT env entry, and things work without a hitch.
(22:20:11) agaffney: that's testing the "wrong" thing, since it wouldn't re-gather that particular fact
(22:20:19) agaffney: you'd need to disable the cache all together to really test that theory
(22:20:34) cognifloyd: Yeah. One sec and I will...
(22:27:55) cognifloyd: Yes, I only get this if it comes from the cache.
(22:30:16) cognifloyd: One playbook run with empty cache gathers facts at the start and caches that, but works just fine. On the next run the conditional 'when: ansible_facts.distribution is not defined' gives the error.
(22:31:17) cognifloyd: If it's already cached, the environment keyword doesn't make a difference.
(22:32:20) agaffney: I wonder if vars lose the AnsibleUnsafe wrapper when being read from the cache
(22:32:32) agaffney: iirc, that prevents templating
(22:34:59) cognifloyd: Hmm. I'll bet my issue (involving cached vars) should be fixed in 2.6: https://github.com/ansible/ansible/commit/cbb6a7f4e831abd2d56375fb0aa22465e0cc7c0c
(22:35:18) cognifloyd: but you said you tested in 2.6 and devel, right?
(22:38:46) agaffney: yes, but couldn't reproduce as I don't use fact caching
(22:40:25) agaffney: I was looking at that code recently, and I don't think it actually works correctly because of the way that JSON decoder works
(22:40:45) agaffney: encoder, not decoder
(22:41:59) cognifloyd: Yep. Something fishy here...
(22:42:20) agaffney: the AnsibleUnsafeText type inherits from the string object, which is unconditionally handled internally by the JSON and coder, so the custom code never gets called
(22:42:33) agaffney: bah, encoder
(22:43:25) cognifloyd: Isn't the else super().default(o) supposed to handle the internal JSON cases?
(22:43:31) agaffney: some code that explicitly calls wrap_var() when reading from the cache would probably solve your problem
(22:44:12) agaffney: iirc, default() only gets called for "unknown" object types
(22:44:28) agaffney: I dug around in this code fairly recently
(22:44:54) cognifloyd: hmm. that would be an issue wouldn't it.
(22:45:05) Frit0ss left the room (quit: Ping timeout: 240 seconds).
(22:45:57) agaffney: you should test with 2.6 and then create an issue, and please CC me on it
(22:46:07) cognifloyd: k
(23:04:58) cognifloyd: Reproduced with 2.6.1
```
As noted in the IRC conversation above, this is probably happening because facts from the cache aren't wrapped in AnsibleUnsafeText, which causes values to be templated. There is code in the JSON encoder to preserve the "unsafe" status of these vars, but it doesn't seem to actually work due to the way the JSON encoder works.

A possible solution is to unconditionally use `ansible.utils.unsafe_proxy.wrap_var()` on facts coming from the cache, but this will also wrap cacheable `set_fact` vars, which wouldn't normally be wrapped.

Resolving patch:
diff --git a/lib/ansible/vars/manager.py b/lib/ansible/vars/manager.py
--- a/lib/ansible/vars/manager.py
+++ b/lib/ansible/vars/manager.py
@@ -308,7 +308,7 @@ def plugins_by_groups():
 
             # finally, the facts caches for this host, if it exists
             try:
-                facts = self._fact_cache.get(host.name, {})
+                facts = wrap_var(self._fact_cache.get(host.name, {}))
                 all_vars.update(namespace_facts(facts))
 
                 # push facts to main namespace


Test patch: