Repository: conda/conda-build

Issue:
run_exports not applied when channel does not provide channeldata.json
As of #3561, run exports are extracted from channeldata.json rather than the packages themselves.  This speeds up rendering time but will cause run exports to not be applied if the channel from which the package is supplied does not provide channeldata.json.  Most channels on Anaconda.org do not provide channeldata.json.

A simple example:

```
package:
  name: exporty
  version: 1.0.0

build:
  run_exports:
    strong:
      - exporty
```

```
package:
  name: example
  version: 1.0.0

requirements:
  build:
    - exporty
```

Building `exporty` and locally and rendering `example` gives a correct rendering with run_export applied:

```
$ conda render example/
--------------
Hash contents:
--------------
{}
----------
meta.yaml:
----------
package:
    name: example
    version: 1.0.0
requirements:
    build:
        - exporty 1.0.0 0
    run:
        - exporty
extra:
    copy_test_source_files: true
    final: true
```

Using the uploaded `exporty` package on Anaconda.org results in no run_exports applied:

```
$ conda render -c jjhelmus example/
--------------
Hash contents:
--------------
{}
----------
meta.yaml:
----------
package:
    name: example
    version: 1.0.0
requirements:
    build:
        - exporty 1.0.0 0
extra:
    copy_test_source_files: true
    final: true
```


Discussion:
@jjhelmus is this why we are missing run_exports for our conda-forge aarch64 and ppc64le packages?
> @jjhelmus is this why we are missing run_exports for our conda-forge aarch64 and ppc64le packages?

Yes, moving the effected conda-build aarch64 packages (3.18.6  3.18.5 3.18.4) to a broken label may be the easiest fix until this is changed in conda-build itself.
There's something wrong with run_exports on linux-64 as well. When I run `conda render` on a package that has `fftw` as a host dependency, it's getting run_exports coming from builds of packages not in the host environment. Seems like something's amiss in general in the run_exports computation.

meta.yaml:

<details>

```yaml
$ cat test/meta.yaml
package:
  name: testing
  version: 1.0

source:

build:
  number: 1

requirements:
  build: []
  host:
    - fftw
  run:
    - fftw

```
</details>

`conda render --verbose test`

<details>

```yaml
Adding in variants from internal_defaults
No numpy version specified in conda_build_config.yaml.  Falling back to default numpy value of 1.11
Attempting to finalize metadata for testing
Collecting package metadata (repodata.json): ...working... done
Solving environment: ...working... done
Collecting package metadata (repodata.json): ...working... done
Solving environment: ...working... done
--------------
Hash contents:
--------------
{}
----------
meta.yaml:
----------
package:
    name: testing
    version: '1.0'
build:
    number: '1'
requirements:
    host:
        - libgcc-ng 9.1.0 hdf63c60_0
        - libgfortran-ng 7.3.0 hdf63c60_0
        - fftw 3.3.8 nompi_h7f3a6c3_1105
    run:
        - fftw * mpi_openmpi_*
extra:
    copy_test_source_files: true
    final: true
```

</details>


`conda info`

<details>

```
     active environment : base
    active env location : /opt/conda
            shell level : 1
       user config file : /home/conda/.condarc
 populated config files : /home/conda/.condarc
          conda version : 4.7.5
    conda-build version : 3.18.6
         python version : 3.7.3.final.0
       virtual packages :
       base environment : /opt/conda  (writable)
           channel URLs : https://conda.anaconda.org/conda-forge/linux-64
                          https://conda.anaconda.org/conda-forge/noarch
                          https://repo.anaconda.com/pkgs/main/linux-64
                          https://repo.anaconda.com/pkgs/main/noarch
                          https://repo.anaconda.com/pkgs/r/linux-64
                          https://repo.anaconda.com/pkgs/r/noarch
          package cache : /opt/conda/pkgs
                          /home/conda/.conda/pkgs
       envs directories : /opt/conda/envs
                          /home/conda/.conda/envs
               platform : linux-64
             user-agent : conda/4.7.5 requests/2.22.0 CPython/3.7.3 Linux/4.9.125-linuxkit centos/6.10 glibc/2.12
                UID:GID : 9001:9001
             netrc file : None
           offline mode : False
```

</details>

FWIW, I confirmed that this does not happen with conda-build 3.18.3 and does with 3.18.4-6.
For further detail about what might make this case weird: there are three builds of fftw 3.3.8: `nompi_`, which has no run_exports, and 2x `mpi_{provider}` which have `run_exports: fftw * mpi_{provider}_*`. I'm guessing the new logic in #3561 is merging them somehow, or clobbering the one empty run_exports with one of the others.
Sadly, we released over 20 packages on the xeus stack making use of the broken version of conda build unfortunately, and there are as many binders that are broken.
Maybe it would be worth your time to add tests to the conda forge
infrastructure or whatever you use so that this doesn't bite you again. Or
perhaps a test in conda build's test suite?

On Wed, Jul 3, 2019, 12:19 Sylvain Corlay <notifications@github.com> wrote:

> Sadly, we released over 20 packages on the xeus stack making use of the
> broken version of conda build unfortunately, and there are as many binders
> that are broken.
>
> —
> You are receiving this because you are subscribed to this thread.
> Reply to this email directly, view it on GitHub
> <https://github.com/conda/conda-build/issues/3583?email_source=notifications&email_token=AAAJL6MDBOUNJUGRMI6K7PLP5TNRVA5CNFSM4H4IVTPKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGODZFD7HQ#issuecomment-508182430>,
> or mute the thread
> <https://github.com/notifications/unsubscribe-auth/AAAJL6NXA4XAIVFZLNVARSDP5TNRVANCNFSM4H4IVTPA>
> .
>

We test that artefacts are properly produced. Although run-exports being completely ignored looks like a rather major issue for a release...
But you apparently don't test that libraries are correctly loaded at test time - if you did, you would have caught this.

Run-exports is not completely ignored.  It works fine for channels that have channeldata.json.  It is tested, and those tests pass.  Since you depend on a channel that does not have channeldata.json, perhaps it is worth your time to invest time in conda-build so that it won't break for you in the future.
Yeah we are not testing the produced runtime but that should be easily fixed (trying to launch the kernel at least if not running the test suite).

Investing into conda-build core at the moment would really hard in terms of bandwidth for the team.
I have never asked you to fix it. I have asked you to help write tests for functionality that you depend on, especially when our tests cover our user cases, but not yours.
>  I have asked you to help write tests for functionality that you depend on.

Sounds good. Would be happy to add such tests!
Testing issues like this would be helped a lot by conda-build supporting multiple test envs. For instance, this issue snuck through on mpich because the test environment had the compiler packages, which included the dependencies missing from run_exports.  It would also enable testing e.g. ABI compatibility by running tests against latest and oldest allowed versions of dependencies, which are currently not testable.

Resolving patch:
diff --git a/conda_build/build.py b/conda_build/build.py
--- a/conda_build/build.py
+++ b/conda_build/build.py
@@ -1058,7 +1058,7 @@ def bundle_conda(output, metadata, env, stats, **kw):
             utils.copy_into(tmp_path, final_output, metadata.config.timeout,
                             locking=False)
             final_outputs.append(final_output)
-    update_index(os.path.dirname(output_folder), verbose=metadata.config.debug)
+    update_index(os.path.dirname(output_folder), verbose=metadata.config.debug, threads=1)
 
     # clean out host prefix so that this output's files don't interfere with other outputs
     #   We have a backup of how things were before any output scripts ran.  That's
@@ -1754,7 +1754,7 @@ def _construct_metadata_for_test_from_package(package, config):
     local_channel = os.path.dirname(local_pkg_location)
 
     # update indices in the channel
-    update_index(local_channel, verbose=config.debug)
+    update_index(local_channel, verbose=config.debug, threads=1)
 
     try:
         metadata = render_recipe(os.path.join(info_dir, 'recipe'), config=config,
@@ -2200,7 +2200,7 @@ def tests_failed(package_or_metadata, move_broken, broken_dir, config):
                     broken_dir))
         except OSError:
             pass
-        update_index(os.path.dirname(os.path.dirname(pkg)), verbose=config.debug)
+        update_index(os.path.dirname(os.path.dirname(pkg)), verbose=config.debug, threads=1)
     sys.exit("TESTS FAILED: " + os.path.basename(pkg))
 
 
@@ -2591,7 +2591,7 @@ def is_package_built(metadata, env, include_local=True):
     for d in metadata.config.bldpkgs_dirs:
         if not os.path.isdir(d):
             os.makedirs(d)
-        update_index(d, verbose=metadata.config.debug, warn=False)
+        update_index(d, verbose=metadata.config.debug, warn=False, threads=1)
     subdir = getattr(metadata.config, '{}_subdir'.format(env))
 
     urls = [url_path(metadata.config.output_folder), 'local'] if include_local else []
diff --git a/conda_build/render.py b/conda_build/render.py
--- a/conda_build/render.py
+++ b/conda_build/render.py
@@ -287,10 +287,11 @@ def execute_download_actions(m, actions, env, package_subset=None, require_files
                                   timeout=m.config.timeout)
 
     # this should be just downloading packages.  We don't need to extract them -
-    #    we read contents directly
+
+    download_actions = {k: v for k, v in actions.items() if k in ('FETCH', 'EXTRACT', 'PREFIX')}
     if 'FETCH' in actions or 'EXTRACT' in actions:
         # this is to force the download
-        execute_actions(actions, index, verbose=m.config.debug)
+        execute_actions(download_actions, index, verbose=m.config.debug)
 
     pkg_files = {}
 
@@ -353,9 +354,19 @@ def get_upstream_pins(m, actions, env):
 
     ignore_list = utils.ensure_list(m.get_value('build/ignore_run_exports'))
     additional_specs = {}
+    run_exports = {}
+    empty_run_exports = False
     for pkg in linked_packages:
         channeldata = utils.download_channeldata(pkg.channel)
-        run_exports = channeldata.get('packages', {}).get(pkg.name, {}).get('run_exports', {}).get(pkg.version, {})
+        if channeldata:
+            pkg_data = channeldata.get('packages', {}).get(pkg.name, {})
+            run_exports = pkg_data.get('run_exports', {}).get(pkg.version, {})
+            empty_run_exports = run_exports == {}
+        if not run_exports and not empty_run_exports:
+            locs_and_dists = execute_download_actions(m, actions, env=env,
+                                                      package_subset=linked_packages)
+            locs_and_dists = [v for k, v in locs_and_dists.items() if k == pkg]
+            run_exports = _read_specs_from_package(*next(iter(locs_and_dists)))
         specs = _filter_run_exports(run_exports, ignore_list)
         if specs:
             additional_specs = utils.merge_dicts_of_lists(additional_specs, specs)


Test patch:
diff --git a/tests/test_api_build.py b/tests/test_api_build.py
--- a/tests/test_api_build.py
+++ b/tests/test_api_build.py
@@ -276,7 +276,7 @@ def test_checkout_tool_as_dependency(testing_workdir, testing_config, monkeypatc
     exename = dummy_executable(testing_workdir, "svn")
     monkeypatch.setenv("PATH", testing_workdir, prepend=os.pathsep)
     FNULL = open(os.devnull, 'w')
-    with pytest.raises(subprocess.CalledProcessError, message="Dummy svn was not executed"):
+    with pytest.raises(subprocess.CalledProcessError):
         check_call_env([exename, '--version'], stderr=FNULL)
     FNULL.close()
     env = os.environ.copy()
@@ -327,32 +327,28 @@ def test_cmake_generator(platform, target_compiler, testing_workdir, testing_con
 
 @pytest.mark.skipif(sys.platform == "win32",
                     reason="No windows symlinks")
-def test_symlink_fail(testing_workdir, testing_config, capfd):
+def test_symlink_fail(testing_workdir, testing_config):
     with pytest.raises((SystemExit, FileNotFoundError)):
         api.build(os.path.join(fail_dir, "symlinks"), config=testing_config)
-    # output, error = capfd.readouterr()
-    # assert error.count("Error") == 6, "did not find appropriate count of Error in: " + error
 
 
 def test_pip_in_meta_yaml_fail(testing_workdir, testing_config):
-    with pytest.raises(ValueError) as exc:
+    with pytest.raises(ValueError, match='environment.yml'):
         api.build(os.path.join(fail_dir, "pip_reqs_fail_informatively"), config=testing_config)
-    assert "environment.yml" in str(exc)
 
 
 def test_recursive_fail(testing_workdir, testing_config):
-    with pytest.raises((RuntimeError, exceptions.DependencyNeedsBuildingError)) as exc:
+    with pytest.raises((RuntimeError, exceptions.DependencyNeedsBuildingError),
+                       match="recursive-build2"):
         api.build(os.path.join(fail_dir, "recursive-build"), config=testing_config)
     # indentation critical here.  If you indent this, and the exception is not raised, then
     #     the exc variable here isn't really completely created and shows really strange errors:
     #     AttributeError: 'ExceptionInfo' object has no attribute 'typename'
-    assert "recursive-build2" in str(exc.value)
 
 
 def test_jinja_typo(testing_workdir, testing_config):
-    with pytest.raises(SystemExit) as exc:
+    with pytest.raises(SystemExit, match="GIT_DSECRIBE_TAG") as exc:
         api.build(os.path.join(fail_dir, "source_git_jinja2_oops"), config=testing_config)
-    assert "GIT_DSECRIBE_TAG" in exc.exconly()
 
 
 def test_skip_existing(testing_workdir, testing_config, capfd):
@@ -387,9 +383,8 @@ def test_skip_existing_url(testing_metadata, testing_workdir, capfd):
 
 def test_failed_tests_exit_build(testing_workdir, testing_config):
     """https://github.com/conda/conda-build/issues/1112"""
-    with pytest.raises(SystemExit) as exc:
+    with pytest.raises(SystemExit, match='TESTS FAILED') as exc:
         api.build(os.path.join(metadata_dir, "_test_failed_test_exits"), config=testing_config)
-    assert 'TESTS FAILED' in str(exc)
 
 
 def test_requirements_txt_for_run_reqs(testing_workdir, testing_config):
@@ -495,7 +490,7 @@ def test_relative_git_url_submodule_clone(testing_workdir, testing_config, monke
     # Strangely ..
     #   stderr=FNULL suppresses the output from echo on OS X whereas
     #   stdout=FNULL suppresses the output from echo on Windows
-    with pytest.raises(subprocess.CalledProcessError, message="Dummy git was not executed"):
+    with pytest.raises(subprocess.CalledProcessError):
         check_call_env([exename, '--version'], stdout=FNULL, stderr=FNULL)
     FNULL.close()
 
diff --git a/tests/test_api_render.py b/tests/test_api_render.py
--- a/tests/test_api_render.py
+++ b/tests/test_api_render.py
@@ -5,6 +5,7 @@
 
 import os
 import re
+import sys
 
 import mock
 import pytest
@@ -218,3 +219,12 @@ def test_merge_build_host_build_key(testing_workdir, testing_metadata):
 def test_merge_build_host_empty_host_section(testing_config):
     m = api.render(os.path.join(metadata_dir, '_empty_host_avoids_merge'))[0][0]
     assert not any('bzip2' in dep for dep in m.meta['requirements']['run'])
+
+
+@pytest.mark.skipif(sys.platform != "linux2", reason="package on remote end is only on linux")
+def test_run_exports_from_repo_without_channeldata(testing_config):
+    ms = api.render(os.path.join(metadata_dir, '_run_export_no_channeldata'), config=testing_config)
+    assert ms[0][0].meta['requirements']['build'] == ["exporty"]
+    # these two will be missing if run_exports has failed.
+    assert ms[0][0].meta['requirements']['host'] == ["exporty"]
+    assert ms[0][0].meta['requirements']['run'] == ["exporty"]
diff --git a/tests/test_post.py b/tests/test_post.py
--- a/tests/test_post.py
+++ b/tests/test_post.py
@@ -45,9 +45,8 @@ def test_postbuild_files_raise(testing_metadata, testing_workdir):
         with open(os.path.join(testing_metadata.config.work_dir,
                                '__conda_{}__.txt'.format(f)), 'w') as fh:
             fh.write('123')
-        with pytest.raises(ValueError) as exc:
+        with pytest.raises(ValueError, match=f) as exc:
             post.get_build_metadata(testing_metadata)
-        assert f in str(exc)
 
 
 @pytest.mark.skipif(on_win, reason="fix_shebang is not executed on win32")
diff --git a/tests/test_render.py b/tests/test_render.py
--- a/tests/test_render.py
+++ b/tests/test_render.py
@@ -1,7 +1,10 @@
 import os
+import sys
 from conda_build import api
 from conda_build import render
 
+import pytest
+
 
 def test_output_with_noarch_says_noarch(testing_metadata):
     testing_metadata.meta['build']['noarch'] = 'python'
diff --git a/tests/test_utils.py b/tests/test_utils.py
--- a/tests/test_utils.py
+++ b/tests/test_utils.py
@@ -313,7 +313,7 @@ def test_try_acquire_locks(testing_workdir):
     lock1_copy = filelock.FileLock(os.path.join(testing_workdir, 'lock1'))
     # Also verify that the error message contains the word "lock", since we rely
     # on this elsewhere.
-    with pytest.raises(BuildLockError, message='Failed to acquire all locks'):
+    with pytest.raises(BuildLockError, match='Failed to acquire all locks'):
         with utils.try_acquire_locks([lock1, lock1_copy], timeout=1):
             pass