Repository: easybuilders/easybuild-framework

Issue:
pyshortver deferencing fails when Python is not a full dependency
I was testing an easyconfig with `Python` as a builddependency, not a full dependency.

In `postinstallcmds`, I tried to use `pyshortver` but this wasn't resolved and instead was passed as a raw value, causing a crash.

This should be picked up earlier, because it's a somewhat problematic configuration anyway.


Discussion:
The problem went away when I instead had `Python` as a full dependency.
@verdurin What do you think the best approach would be here? Try and scan easyconfigs for templates that we won't be able to resolve at all?

The tricky bit is that some templates can only be resolved as a certain point in time...
See also #3300, basically same issue (but there with a use case that shows that filtering out build dependencies probably doesn't make sense).

Resolving patch:
diff --git a/easybuild/framework/easyconfig/easyconfig.py b/easybuild/framework/easyconfig/easyconfig.py
--- a/easybuild/framework/easyconfig/easyconfig.py
+++ b/easybuild/framework/easyconfig/easyconfig.py
@@ -1593,6 +1593,7 @@ def _finalize_dependencies(self):
     def generate_template_values(self):
         """Try to generate all template values."""
 
+        self.log.info("Generating template values...")
         self._generate_template_values()
 
         # recursive call, until there are no more changes to template values;
@@ -1611,6 +1612,8 @@ def generate_template_values(self):
                     # KeyError's may occur when not all templates are defined yet, but these are safe to ignore
                     pass
 
+        self.log.info("Template values: %s", ', '.join("%s='%s'" % x for x in sorted(self.template_values.items())))
+
     def _generate_template_values(self, ignore=None):
         """Actual code to generate the template values"""
 
diff --git a/easybuild/framework/easyconfig/templates.py b/easybuild/framework/easyconfig/templates.py
--- a/easybuild/framework/easyconfig/templates.py
+++ b/easybuild/framework/easyconfig/templates.py
@@ -216,9 +216,25 @@ def template_constant_dict(config, ignore=None, skip_lower=None, toolchain=None)
         # copy to avoid changing original list below
         deps = copy.copy(config.get('dependencies', []))
 
-        # only consider build dependencies for defining *ver and *shortver templates if we're in iterative mode
-        if hasattr(config, 'iterating') and config.iterating:
-            deps += config.get('builddependencies', [])
+        # also consider build dependencies for *ver and *shortver templates;
+        # we need to be a bit careful here, because for iterative installations
+        # (when multi_deps is used for example) the builddependencies value may be a list of lists
+
+        # first, determine if we have an EasyConfig instance
+        # (indirectly by checking for 'iterating' and 'iterate_options' attributes,
+        #  because we can't import the EasyConfig class here without introducing
+        #  a cyclic import...);
+        # we need to know to determine whether we're iterating over a list of build dependencies
+        is_easyconfig = hasattr(config, 'iterating') and hasattr(config, 'iterate_options')
+
+        if is_easyconfig:
+            # if we're iterating over different lists of build dependencies,
+            # only consider build dependencies when we're actually in iterative mode!
+            if 'builddependencies' in config.iterate_options:
+                if config.iterating:
+                    deps += config.get('builddependencies', [])
+            else:
+                deps += config.get('builddependencies', [])
 
         for dep in deps:
             if isinstance(dep, dict):


Test patch:
diff --git a/test/framework/easyconfig.py b/test/framework/easyconfig.py
--- a/test/framework/easyconfig.py
+++ b/test/framework/easyconfig.py
@@ -965,6 +965,8 @@ def test_templating(self):
             '   ("Java", "1.7.80"),'
             '   ("Perl", "5.22.0"),'
             '   ("Python", "2.7.10"),'
+            ']',
+            'builddependencies = ['
             '   ("R", "3.2.3"),'
             ']',
             'modloadmsg = "%s"' % '; '.join([