Repository: conda/conda-build

Issue:
blas *-devel libraries generating additional build variants
This is a duplicate of #7630 on Anaconda/issues, but I've closed that and opened here on the advice of @msarahan. I'm using conda 4.3.30.

Here's a toy example:

meta.yaml:
``` yaml
package:
        name: test
        version: 0.0

source:
        path: .

requirements:
        build:
{% if blas_impl == 'mkl' %}
                - mkl-devel
{% endif %}
{% if blas_impl == 'openblas' %}
                - openblas-devel
{% endif %}
```
conda_build_config.yaml
``` yaml
blas_impl:
        - mkl
        - openblas
```

Using the ``print_yaml`` code from the [conda build 3 blog post](https://www.anaconda.com/blog/developer-blog/package-better-conda-build-3/):
```
>>> print_yamls('dumb_recipe')
package:
    name: test
    version: '0.0'
source:
    path: /Users/cl_mac/Projects/Code/pydrobert/kaldi/dumb_recipe
requirements:
    build:
        - mkl-include 2018.0.1 h8158457_4
        - mkl 2018.0.1 hfbd8650_4
        - mkl-devel 2018.0.1 h5cfdff2_4
        - intel-openmp 2018.0.0 h8158457_8
    run:
        - mkl >=2018.0.1
extra:
    copy_test_source_files: true
    final: true

--------------------------------------------------
package:
    name: test
    version: '0.0'
source:
    path: /Users/cl_mac/Projects/Code/pydrobert/kaldi/dumb_recipe
requirements:
    build:
        - libopenblas 0.2.20 h6c53463_3
        - libgfortran 3.0.1 h93005f0_2
        - openblas-devel 0.2.20 h1e4b922_3
extra:
    copy_test_source_files: true
    final: true

--------------------------------------------------
package:
    name: test
    version: '0.0'
source:
    path: /Users/cl_mac/Projects/Code/pydrobert/kaldi/dumb_recipe
requirements:
    build:
        - mkl-include 2018.0.1 h8158457_4
        - mkl 2018.0.1 hfbd8650_4
        - mkl-devel 2018.0.1 h5cfdff2_4
        - intel-openmp 2018.0.0 h8158457_8
extra:
    copy_test_source_files: true
    final: true

--------------------------------------------------
package:
    name: test
    version: '0.0'
source:
    path: /Users/cl_mac/Projects/Code/pydrobert/kaldi/dumb_recipe
requirements:
    build:
        - libopenblas 0.2.20 h6c53463_3
        - libgfortran 3.0.1 h93005f0_2
        - openblas-devel 0.2.20 h1e4b922_3
    run:
        - libopenblas >=0.2.20,<0.2.21.0a0
extra:
    copy_test_source_files: true
    final: true

--------------------------------------------------
```
As you can see, the build matrix is of size 4 instead of 2. The additional matrices appear to derive from either including or excluding the library in the list of runtime dependencies. Replacing ``mkl-devel`` with ``mkl`` and ``mkl-include`` drops the matrix to one less, and replacing ``openblas-devel`` with ``openblas`` drops it as well. If you make both changes, you get:
```
>>> print_yamls('dumb_recipe')
package:
    name: test
    version: '0.0'
source:
    path: /Users/cl_mac/Projects/Code/pydrobert/kaldi/dumb_recipe
requirements:
    build:
        - mkl-include 2018.0.1 h8158457_4
        - mkl 2018.0.1 hfbd8650_4
        - intel-openmp 2018.0.0 h8158457_8
extra:
    copy_test_source_files: true
    final: true

--------------------------------------------------
package:
    name: test
    version: '0.0'
source:
    path: /Users/cl_mac/Projects/Code/pydrobert/kaldi/dumb_recipe
requirements:
    build:
        - libgfortran 3.0.1 h93005f0_2
        - libgcc 4.8.5 hdbeacc1_10
        - openblas 0.2.20 ha880b72_0
extra:
    copy_test_source_files: true
    final: true

--------------------------------------------------
```
Which does not contain the entries with runtime dependencies. This is not a great workaround because it decouples the versions of the include files and shared libraries.

As for the run exports, if the associated library is manually included in the runtime requirements, you can get some funny duplication:
```
run:
        - libopenblas
        - libopenblas >=0.2.20,<0.2.21.0a0
```
probably harmless, though.

Thanks,
Sean


Discussion:
I'm not sure why this happens.  I recommend adjust your recipe to the following, which fixes it for me:

```
package:
  name: test
  version: 0.0

source:
  path: .

requirements:
  build:
    - {{ blas_impl }}-devel
```

With conda-build 3.1.0, the simplified hashes cut out the variant completely because conda-build doesn't seem to be able to match up against stuff in the if statements.  You end up with only one output.  I can try to fix that.  I guess it's just a matter of tweaking regexes.  Really, you should prefer selectors over jinja2 blocks for this, though.

Resolving patch:
diff --git a/conda_build/variants.py b/conda_build/variants.py
--- a/conda_build/variants.py
+++ b/conda_build/variants.py
@@ -487,8 +487,9 @@ def find_used_variables_in_text(variant, recipe_text):
     used_variables = set()
     for v in variant:
         variant_regex = r"(\s*\{\{\s*%s\s*(?:.*?)?\}\})" % v
+        conditional_regex = r"(\s*\{%\s*(?:el)?if\s*" + v + r"\s*(?:.*?)?%\})"
         requirement_regex = r"(\-\s+%s(?:\s+|$))" % v.replace('_', '[-_]')
-        all_res = '|'.join((variant_regex, requirement_regex))
+        all_res = '|'.join((variant_regex, requirement_regex, conditional_regex))
         compiler_match = re.match(r'(.*?)_compiler$', v)
         if compiler_match:
             compiler_regex = (


Test patch:
diff --git a/tests/test-recipes/variants/21_conditional_sections/conda_build_config.yaml b/tests/test-recipes/variants/21_conditional_sections/conda_build_config.yaml
new file mode 100644
--- /dev/null
+++ b/tests/test-recipes/variants/21_conditional_sections/conda_build_config.yaml
@@ -0,0 +1,3 @@
+blas_impl:
+  - mkl
+  - openblas
diff --git a/tests/test-recipes/variants/21_conditional_sections/meta.yaml b/tests/test-recipes/variants/21_conditional_sections/meta.yaml
new file mode 100644
--- /dev/null
+++ b/tests/test-recipes/variants/21_conditional_sections/meta.yaml
@@ -0,0 +1,15 @@
+package:
+  name: test
+  version: 0.0
+
+source:
+  path: .
+
+requirements:
+  build:
+{% if blas_impl == 'mkl' %}
+    - mkl-devel
+{% endif %}
+{% if blas_impl == 'openblas' %}
+    - openblas-devel
+{% endif %}
diff --git a/tests/test_variants.py b/tests/test_variants.py
--- a/tests/test_variants.py
+++ b/tests/test_variants.py
@@ -276,3 +276,11 @@ def test_reduced_hashing_behavior(testing_config):
     m = api.render(os.path.join(recipe_dir, '02_python_version'), finalize=False, bypass_env_check=True)[0][0]
     assert not m.get_hash_contents()
     assert not re.search('h[0-9a-f]{%d}' % testing_config.hash_length, m.build_id())
+
+
+def test_variants_used_in_jinja2_conditionals(testing_config):
+    ms = api.render(os.path.join(recipe_dir, '21_conditional_sections'),
+                    finalize=False, bypass_env_check=True)
+    assert len(ms) == 2
+    assert sum(m.config.variant['blas_impl'] == 'mkl' for m, _, _ in ms) == 1
+    assert sum(m.config.variant['blas_impl'] == 'openblas' for m, _, _ in ms) == 1