Repository: conda/conda-build

Issue:
pinnings not applied to top level requirements for recipes with outputs
Pinnings from variant files are not applied to the top level requirements in recipes:

For example

**meta.yaml**
``` yaml
package:
  name: top_level
  version: 1.0

requirements:
  host:
    - bzip2

outputs:
  - name: example
    requirements:
      host:
        - bzip2
```

**var.yaml**
``` yaml
bzip2:
  - 1.0.6
```

``` console
$ conda build --version
conda-build 3.18.8
$ conda build -m var.yaml .
No numpy version specified in conda_build_config.yaml.  Falling back to default numpy value of 1.11
WARNING:conda_build.metadata:No numpy version specified in conda_build_config.yaml.  Falling back to default numpy value of 1.11
Adding in variants from internal_defaults
INFO:conda_build.variants:Adding in variants from internal_defaults
Adding in variants from var.yaml
INFO:conda_build.variants:Adding in variants from var.yaml
Attempting to finalize metadata for example
INFO:conda_build.metadata:Attempting to finalize metadata for example
Collecting package metadata (repodata.json): ...working... done
Solving environment: ...working... done
Collecting package metadata (repodata.json): ...working... done
Solving environment: ...working... done
BUILD START: ['example-1.0-h77a7685_0.tar.bz2']
Collecting package metadata (repodata.json): ...working... done
Solving environment: ...working... done

## Package Plan ##

  environment location: /home/jhelmus/conda/conda-bld/top_level_1563489606679/_h_env_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_plac


The following NEW packages will be INSTALLED:

    _libgcc_mutex: 0.1-main         defaults
    bzip2:         1.0.8-h7b6447c_0 defaults
    libgcc-ng:     9.1.0-hdf63c60_0 defaults
...
```



Discussion:
This example worked with conda build 3.17.8

```console
$ conda build --version
conda-build 3.17.8
$ conda build -m var.yaml .
No numpy version specified in conda_build_config.yaml.  Falling back to default numpy value of 1.11
WARNING:conda_build.metadata:No numpy version specified in conda_build_config.yaml.  Falling back to default numpy value of 1.11
Adding in variants from internal_defaults   
INFO:conda_build.variants:Adding in variants from internal_defaults
Adding in variants from var.yaml
INFO:conda_build.variants:Adding in variants from var.yaml
Attempting to finalize metadata for example
INFO:conda_build.metadata:Attempting to finalize metadata for example
WARNING:conda.base.context:Conda is constrained to only using the old .tar.bz2 file format because you have conda-build installed, and it is <3.18.3.  Update or remove conda-build to get smaller download
s and faster extractions.                                      
WARNING conda.base.context:use_only_tar_bz2(632): Conda is constrained to only using the old .tar.bz2 file format because you have conda-build installed, and it is <3.18.3.  Update or remove conda-build
to get smaller downloads and faster extractions.
Collecting package metadata (repodata.json): ...working... done           
Solving environment: ...working... done                                                                                                                                                                    Collecting package metadata (repodata.json): ...working... done   
Solving environment: ...working... done                                             
BUILD START: ['example-1.0-h77a7685_0.tar.bz2']                          
Collecting package metadata (repodata.json): ...working... done
Solving environment: ...working... done                           
Collecting package metadata (repodata.json): ...working... done
Solving environment: ...working... done                                                                  
                         
## Package Plan ##                                                                                          
                                               
  environment location: /home/jhelmus/conda/conda-bld/top_level_1563489836588/_h_env_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehol
d_placehold_placehold_placehold_placehold_placehold_placehold_placehold_plac    
                                                                                                       
                  
The following NEW packages will be INSTALLED:                                                     
                                   
    _libgcc_mutex: 0.1-main         defaults                                                              
    bzip2:         1.0.6-h14c3975_5 defaults                               
    libgcc-ng:     9.1.0-hdf63c60_0 defaults                     
```
bisects to one of:
ea6529e8169ed9a8aa9efb1e3c08d5a78fc9c34c
f44b207358e0bff726941b910b0f0bfbc716a9fe
6deb528dd0d5ff76df9709f8a6081fe4f8ed189b

f44b207 looks the most likely

Resolving patch:
diff --git a/conda_build/build.py b/conda_build/build.py
--- a/conda_build/build.py
+++ b/conda_build/build.py
@@ -61,7 +61,8 @@
 from conda_build import environ, source, tarcheck, utils
 from conda_build.index import get_build_index, update_index
 from conda_build.render import (output_yaml, bldpkg_path, render_recipe, reparse, distribute_variants,
-                                expand_outputs, try_download, execute_download_actions)
+                                expand_outputs, try_download, execute_download_actions,
+                                add_upstream_pins)
 import conda_build.os_utils.external as external
 from conda_build.metadata import FIELDS, MetaData, default_structs
 from conda_build.post import (post_process, post_build,
@@ -1337,6 +1338,7 @@ def build(m, stats, post=None, need_source_download=True, need_reparse_in_env=Fa
         # TODO: should we check both host and build envs?  These are the same, except when
         #    cross compiling.
         top_level_pkg = m
+        top_level_needs_finalizing = True
         for _, om in output_metas:
             if om.skip() or (m.config.skip_existing and is_package_built(om, 'host')):
                 skipped.append(bldpkg_path(om))
@@ -1344,6 +1346,7 @@ def build(m, stats, post=None, need_source_download=True, need_reparse_in_env=Fa
                 package_locations.append(bldpkg_path(om))
             if om.name() == m.name():
                 top_level_pkg = om
+                top_level_needs_finalizing = False
         if not package_locations:
             print("Packages for ", m.path or m.name(), "with variant {} "
                   "are already built and available from your configured channels "
@@ -1385,6 +1388,22 @@ def build(m, stats, post=None, need_source_download=True, need_reparse_in_env=Fa
                                     " does not yet support Python 3.  Please handle all of "
                                     "your mercurial actions outside of your build script.")
 
+        if top_level_needs_finalizing:
+            utils.insert_variant_versions(
+                top_level_pkg.meta.get('requirements', {}), top_level_pkg.config.variant, 'build')
+            utils.insert_variant_versions(
+                top_level_pkg.meta.get('requirements', {}), top_level_pkg.config.variant, 'host')
+
+            exclude_pattern = None
+            excludes = set(top_level_pkg.config.variant.get('ignore_version', []))
+            for key in top_level_pkg.config.variant.get('pin_run_as_build', {}).keys():
+                if key in excludes:
+                    excludes.remove(key)
+            if excludes:
+                exclude_pattern = re.compile(r'|'.join(r'(?:^{}(?:\s|$|\Z))'.format(exc)
+                                                for exc in excludes))
+            add_upstream_pins(m, False, exclude_pattern)
+
         create_build_envs(top_level_pkg, notest)
 
         # this check happens for the sake of tests, but let's do it before the build so we don't


Test patch:
diff --git a/tests/test-recipes/variants/30_top_level_finalized/conda_build_config.yaml b/tests/test-recipes/variants/30_top_level_finalized/conda_build_config.yaml
new file mode 100644
--- /dev/null
+++ b/tests/test-recipes/variants/30_top_level_finalized/conda_build_config.yaml
@@ -0,0 +1,2 @@
+xz:
+  - 5.2.3
diff --git a/tests/test-recipes/variants/30_top_level_finalized/meta.yaml b/tests/test-recipes/variants/30_top_level_finalized/meta.yaml
new file mode 100644
--- /dev/null
+++ b/tests/test-recipes/variants/30_top_level_finalized/meta.yaml
@@ -0,0 +1,18 @@
+package:
+  name: top_level
+  version: 1.0
+
+build:
+  script: "xzcat --version > ${PREFIX}/xzcat_output"
+
+requirements:
+  host:
+    - xz
+
+outputs:
+  - name: output_level
+    files:
+      - xzcat_output
+    requirements:
+      host:
+        - xz
diff --git a/tests/test_variants.py b/tests/test_variants.py
--- a/tests/test_variants.py
+++ b/tests/test_variants.py
@@ -456,3 +456,12 @@ def test_different_git_vars():
     versions = [m[0].version() for m in ms]
     assert "1.20.0" in versions
     assert "1.21.11" in versions
+
+
+@pytest.mark.skipif(sys.platform != "linux", reason="recipe uses a unix specific script")
+def test_top_level_finalized(testing_config):
+    # see https://github.com/conda/conda-build/issues/3618
+    recipe = os.path.join(recipe_dir, '30_top_level_finalized')
+    outputs = api.build(recipe, config=testing_config)
+    xzcat_output = package_has_file(outputs[0], 'xzcat_output')
+    assert '5.2.3' in xzcat_output