Repository: conda/conda-build

Issue:
skip-existing still doesn't skip existing packages in 3.0.4
After updating to 3.0.4 the crash documented in #2172 is fixed. Unfortunately the build is still not skipping existing packages, it just starts to build.. that functionality worked fine in 2.1.17.


Discussion:
We do have tests for this, but you're hitting a path that seems like some
parts of a package are built, but others are not. Can you post a minimal
reproducing recipe?

On Jul 12, 2017 2:55 AM, "gabm" <notifications@github.com> wrote:

> After updating to 3.0.4 the crash documented in #2172
> <https://github.com/conda/conda-build/issues/2172> is fixed.
> Unfortunately the build is still not skipping existing packages, it just
> starts to build.. that functionality worked fine in 2.1.17.
>
> —
> You are receiving this because you are subscribed to this thread.
> Reply to this email directly, view it on GitHub
> <https://github.com/conda/conda-build/issues/2187>, or mute the thread
> <https://github.com/notifications/unsubscribe-auth/AACV-W7XlYhYOjim5zQiSFBJMjDZ2vREks5sNHwFgaJpZM4OVQa6>
> .
>

well its as easy as:

```
package:
  name: "testpkg"
  version: "1.0.0"
```

just build it twice with skip existing... in conda 3.0.4 it says:

```
$ conda build . --skip-existing
source tree in: /home/gabm/miniconda3/conda-bld/testpkg_1499864529470/work
Attempting to finalize metadata for testpkg
INFO:conda_build.metadata:Attempting to finalize metadata for testpkg
BUILD START: ['testpkg-1.0.0-hbf21a9e_0.tar.bz2']
Packaging testpkg
INFO:conda_build.build:Packaging testpkg
Packaging testpkg-1.0.0-hbf21a9e_0
INFO:conda_build.build:Packaging testpkg-1.0.0-hbf21a9e_0
number of files: 0
Fixing permissions
Fixing permissions
updating: testpkg-1.0.0-hbf21a9e_0.tar.bz2
Nothing to test for: /home/gabm/miniconda3/conda-bld/linux-64/testpkg-1.0.0-hbf21a9e_0.tar.bz2
# Automatic uploading is disabled
# If you want to upload package(s) to anaconda.org later, type:

anaconda upload /home/gabm/miniconda3/conda-bld/linux-64/testpkg-1.0.0-hbf21a9e_0.tar.bz2

# To have conda build upload to anaconda.org automatically, use
# $ conda config --set anaconda_upload yes

anaconda_upload is not set.  Not uploading wheels: []



####################################################################################
Source and build intermediates have been left in /home/gabm/miniconda3/conda-bld.
There are currently 85 accumulated.
To remove them, you can run the ```conda build purge``` command
```

while in 2.1.17 it says:

```
 gabm@pc-klosi /t/test  $ conda build . --skip-existing
updating index in: /home/gabm/miniconda3/conda-bld/linux-64
updating index in: /home/gabm/miniconda3/conda-bld/noarch
Fetching package metadata .................
testpkg-1.0.0-0 is already built in ['local'], skipping.
INFO:conda_build.config:--dirty flag and --keep-old-work not specified.Removing build/test folder after successful build/test.

# Automatic uploading is disabled
# If you want to upload package(s) to anaconda.org later, type:


# To have conda build upload to anaconda.org automatically, use
# $ conda config --set anaconda_upload yes

anaconda_upload is not set.  Not uploading wheels: []



####################################################################################
Source and build intermediates have been left in /home/gabm/miniconda3/conda-bld.
There are currently 85 accumulated.
To remove them, you can run the ```conda build purge``` command
```


I simulated it with local, but in our real usecase its a package in a remote repo - same effect
OK, thanks.  That's basically what our test is, so something is wrong there.  I'll look into it.

Resolving patch:
diff --git a/conda_build/build.py b/conda_build/build.py
--- a/conda_build/build.py
+++ b/conda_build/build.py
@@ -1816,11 +1816,11 @@ def is_package_built(metadata, env):
                                       locking=metadata.config.locking,
                                       timeout=metadata.config.timeout,
                                       clear_cache=True)
-    urls = [url_path(metadata.config.croot)] + get_rc_urls()
+    urls = [url_path(metadata.config.output_folder)] + get_rc_urls()
     if metadata.config.channel_urls:
         urls.extend(metadata.config.channel_urls)
 
     # will be empty if none found, and evalute to False
     found_urls = [url for url in urls
-            if dist_str_in_index(index, url + '::' + metadata.pkg_fn())]
+            if dist_str_in_index(index, url + '::' + metadata.dist())]
     return found_urls[0] if found_urls else None
diff --git a/conda_build/config.py b/conda_build/config.py
--- a/conda_build/config.py
+++ b/conda_build/config.py
@@ -70,7 +70,7 @@ def _ensure_dir(path):
             Setting('timeout', 90),
             Setting('set_build_id', True),
             Setting('disable_pip', False),
-            Setting('output_folder', None),
+            Setting('_output_folder', None),
             Setting('prefix_length_fallback', True),
             Setting('_prefix_length', DEFAULT_PREFIX_LENGTH),
             Setting('long_test_prefix', True),
@@ -298,6 +298,14 @@ def croot(self, croot):
         """Set croot - if None is passed, then the default value will be used"""
         self._croot = croot
 
+    @property
+    def output_folder(self):
+        return self._output_folder or self.croot
+
+    @output_folder.setter
+    def output_folder(self, value):
+        self._output_folder = value
+
     @property
     def build_folder(self):
         """This is the core folder for a given build.


Test patch:
diff --git a/tests/test_api_build.py b/tests/test_api_build.py
--- a/tests/test_api_build.py
+++ b/tests/test_api_build.py
@@ -16,8 +16,7 @@
 # for version
 import conda
 
-from conda_build.conda_interface import PY3, url_path, LinkError, CondaError
-from conda_build.conda_interface import PY3, url_path, cc_conda_build
+from conda_build.conda_interface import PY3, url_path, LinkError, CondaError, cc_conda_build
 import conda_build
 
 from binstar_client.commands import remove, show
diff --git a/tests/test_cli.py b/tests/test_cli.py
--- a/tests/test_cli.py
+++ b/tests/test_cli.py
@@ -485,3 +485,15 @@ def test_conda_py_no_period(testing_workdir, testing_metadata, monkeypatch):
     api.output_yaml(testing_metadata, 'meta.yaml')
     outputs = api.build(testing_workdir)
     assert any('py34' in output for output in outputs)
+
+
+@pytest.mark.serial
+def test_build_skip_existing(testing_workdir, testing_config):
+    # build the recipe first
+    empty_sections = os.path.join(metadata_dir, "empty_sections")
+    args = ['--no-anaconda-upload', empty_sections]
+    main_build.execute(args)
+    args.insert(0, '--skip-existing')
+    main_build.execute(args)
+    # output, error = capfd.readouterr()
+    # assert "are already built" in output