Repository: ansible/ansible

Issue:
failing lookup affects copy/template/file even when not part of the task
##### Issue Type:

Bug
##### Ansible Version:

ansible 1.8.2
##### Environment:

debian 7 / gentoo
##### Summary:

The copy and template modules throws an error with a wrong file.
In ansible 1.7.x everything is fine and ansible copies the jail.conf.
In ansible 1.8.2 i cannot copy any file while i have a lookup defined in group vars. Even if the defined lookup is not used in any playbook
##### Steps To Reproduce:

group_vars/all

ssh_users:
- name: evo
  
  key: "{{ lookup('file', 'evo.pub') }}"

tasks/main.yml
- name: copy fail2ban config
  
  copy: src=jail.conf dest=/etc/fail2ban/jail.conf
##### Expected Results:

ansible 1.7.x

TASK: [common-j | copy fail2ban config] ***********************************\* 
changed: [evosrv.test]
##### Actual Results:

ansible 1.8.2

TASK: [common-j | copy fail2ban config] ***********************************\* 
fatal: [evosrv.test] => could not locate file in lookup: evo.pub



Discussion:
Appears I'm having the exact same issue. Same playbook used to work, now aborts early in "ansible-playbook". CentOS 6.6 X86_64 with all updates applied.

<c3p0> ESTABLISH CONNECTION FOR USER: root
fatal: [c3p0] => could not locate file in lookup: ssh_pubkeys/root.pub

Version history:
ansible-1.8.2-1.el6.noarch << ansible-1.8.1-1.el6.noarch << ansible-1.7.2-2.el6.noarch

I'm 99% sure that the same Ansible playbooks worked with 1.8.1.
Cannot "yum downgrade", unfortunately, to be 100% sure.

I have a similar issue, with a completely unrelated lookup in group_vars/all suddenly blowing up whilst doing a synchronize. Downgrading to 1.8.1 fixes the issue. (This has worked fine in many earlier versions.)

Did some experimenting:
 commit e938f554b74c35cc3a13f72bce46ebe5fb3aab3d - FAIL
 commit 2d17d18c1b486a22727b11f6ecae8177e90c76b3 - OK

When checking out earlier commits, playbooks run just fine.

@evoncken so the commit just pushes up an existing error that was swallowed before, trying to avoid the lookup running at all when not used in the template

still the same error in 1.8.4

@evoracer why did yclose the issue? reopening as this still happens

I am sorry. I pushed the wrong button. :/

it happens, thankfully its harder to launch nukes than to close a bug (i hope)

tested v2 and this is solved there

So, @bcoca and I (mainly bcoca :-) took a look at this for a few hours today.  Unfortunately, the code that's causing this is a mess.  In fact, it's one of the sections of code that made us decide to create v2.  The strings that we get from the yaml playbooks need to be templated by jinja2 using the other available variables.  As ansible works through tasks, we potentially define more variables (or overwrite previous ones with new values).  In the v1 codebase, adding these variables and templating are both  being performed multiple times at multiple points in the code on the same variables.  This leads to the situation where modifying how a variable is templated with which variables at one point in the code can have unfortunate side effects on other points in the code.

This issue seems to be the result of just such a sequence of fixes for 1.8.x.  Fixing this in v1 is hard as the fixes will almost certainly cause new bugs to appear and may cause slightly different versions of old bugs to reappear.  The v2 code base cleans up these areas via a VariableManager class that manages the order and templating of variables so fixing these things and understanding the ramifications of the fixes are much better there.  We'll continue to explore the code for this but it may be that other bugs are more important to keep fixed than this one so a fix for this may not be available until v2.

Sorry to be the bearer of discouraging news :-( 


Resolving patch:
diff --git a/lib/ansible/playbook/play.py b/lib/ansible/playbook/play.py
--- a/lib/ansible/playbook/play.py
+++ b/lib/ansible/playbook/play.py
@@ -119,7 +119,11 @@ def __init__(self, playbook, ds, basedir, vault_password=None):
         temp_vars = utils.combine_vars(self.vars, self.vars_file_vars)
         temp_vars = utils.combine_vars(temp_vars, self.playbook.extra_vars)
 
-        ds = template(basedir, ds, temp_vars)
+        try:
+            ds = template(basedir, ds, temp_vars)
+        except errors.AnsibleError, e:
+            utils.warning("non fatal error while trying to template play variables: %s" % (str(e)))
+
         ds['tasks'] = _tasks
         ds['handlers'] = _handlers
 
diff --git a/lib/ansible/utils/template.py b/lib/ansible/utils/template.py
--- a/lib/ansible/utils/template.py
+++ b/lib/ansible/utils/template.py
@@ -118,7 +118,10 @@ def template(basedir, varname, templatevars, lookup_fatal=True, depth=0, expand_
 
         if isinstance(varname, basestring):
             if '{{' in varname or '{%' in varname:
-                varname = template_from_string(basedir, varname, templatevars, fail_on_undefined)
+                try:
+                    varname = template_from_string(basedir, varname, templatevars, fail_on_undefined)
+                except errors.AnsibleError, e:
+                    raise errors.AnsibleError("Failed to template %s: %s" % (varname, str(e)))
 
                 if (varname.startswith("{") and not varname.startswith("{{")) or varname.startswith("["):
                     eval_results = utils.safe_eval(varname, locals=templatevars, include_exceptions=True)


Test patch: