Repository: conan-io/conan

Issue:
Conan install won't call build if a previous one with same build_id failed.
Hi, I'm running into a weird behaviour with Conan. 
When installing a project to my local cache, that had failed during a previous install (with same recipe), Conan won't call the build method and calls the package method directly, even with --build MyProject . 

The workaround is to manually remove the build folder and re run conan install.

I want to know if it's a normal behavior and if there is a way to bypass it.

I appologize if there is already an issue, for the same thing I clearly didn't take enough time to check every issues, and if there is one I'll be glad to know where.

Thank you. 

## Configuration 

- Os : Windows 10
- Arch : x64
- conan version : 1.3.2
- python version : 2.7
- cmake version : 3.11

## Conanfile

```python
from conans import ConanFile, CMake, tools
import shutil
import os
import subprocess

class MyProjectPackage(ConanFile):
    name = "MyProject"
    license = ""
    settings = "os", "compiler", "build_type", "arch"
    generators = "cmake"
    short_paths = True
    exports_sources = "*", "test_package/*", "!conanfile.py", "!.gitignore"
    options = { "myOption" : [True, False]}
    default_options = "myOption=True"

    def build_id(self) : 
        self.info_build.options.myOption = 'Whatever'

    def build(self):
        cmake = CMake(self, generator="Visual Studio 15 2017 Win64")
        args = ['-DCMAKE_CONFIGURATION_TYPES=%s' % self.settings.build_type]
        cmake.configure(args=args)
        cmake.build(args=None, target=None)
        cmake.install(args=None, build_dir=None)

    def package(self):
        self.copy("*", dst="./", src="./EXPORT/" + str(self.settings.build_type), keep_path=True)
```

## How to reproduce

Run : 
```
conan export . MyProject/1.0@user/channel
```
Then
```
conan install MyProject/1.0@user/channel --build MyProject
```
The build failed.
Retry to install to local cache
```
conan install MyProject/1.0@user/channel --build MyProject
```

The build method is skipped, conan calls package method directly.

Here we expect the build method to be called again as with a recipe without build_id method.


Discussion:
Hi @NokiDev !

Thanks for your report. I have been playing with it, changing a few things, as-is is not reproducible, so I have used a "Hello" dependency generated with ``conan new``, and removed the actual build() contents, as the important thing is to check that it is being called or not.

It seems it works as intended here, and the second time the build() is not called at all. If you could please try to put something that is fully reproducible, that could help (like a full github repo, or some recipe and the previous steps to be able to make it work).

Also your default_options is wrong, it should be ``myDep:option=value``, not a dot.

Please try to provide something reproducible and we will try to help. Thanks!
Hi @memsharded !

I created a git repo (Here : https://github.com/NokiDev/conan_build_skipped), to highlight the issue.

Thanks for your time, and have a nice day.
Hi @memsharded,

After a little of investigation, the error seems to be due to this if statement in installer.py > prepare_build()
```python
if self.build_reference != self._package_reference and \
os.path.exists(self.build_folder) and hasattr(self._conan_file, "build_id"):
    self.skip_build=True
    return
```
As the build folder is not removed uppon an error, (and we're using build_id () ) it leads to a skip build. 

We should try to remove it, in the try / except statement in installer.py > _build_package()  if the build failed or keep track of the state of the build in a file.


EDIT : Nop, forget what I was saying it's not there. I'm not familiar enough with conan source code, I suppose. I'll keep searching.
Hi @memsharded, @lasote 
Seems that this issue has been forgotten... 
However the bug is still present with conan 1.8.2

I' updated my repository higlighting the unwanted behaviour. 
(https://github.com/NokiDev/conan_build_skipped)

As a Reminder, the problem was : 
If the build step fails with a conanfile containing build_id method
   Then installation / creation (conan create) of the package won't call build method, and will directly call package method.

Leading to an empty package, and no further builds. 

Tell me if you need something else
Hi @NokiDev 

Sorry for that. I have assigned the issue for next 1.9, I'll try to have a look at it for next release. Thanks for reporting and following up.
I am having a look at this for 1.9. Able to reproduce it, but a note about your code: It is NOT enough to ``return -1`` in the ``build()`` method. That is not an error, and the package creation finish successfully. In order to really fail the build you need to raise an Exception (failed CMake/compile will also raise).

I have been able to reproduce, even raising exceptions, investigating possible fixes.
Okay, 
Thanks for the tip, I thought that return code would be checked by Conan, my bad, I'll update my code.

Glad you be able to reproduce it !

I didn't checked if it produces this kind of issue with package_id method, but it could be great to check it as well.

Resolving patch:
diff --git a/conans/client/installer.py b/conans/client/installer.py
--- a/conans/client/installer.py
+++ b/conans/client/installer.py
@@ -27,7 +27,7 @@
 from conans.paths import CONANINFO, BUILD_INFO, RUN_LOG_NAME
 from conans.util.env_reader import get_env
 from conans.util.files import (save, rmdir, mkdir, make_read_only,
-                               set_dirty, clean_dirty, load)
+                               set_dirty, clean_dirty, load, is_dirty)
 from conans.util.log import logger
 from conans.util.tracer import log_package_built, \
     log_package_got_from_local_cache
@@ -373,14 +373,16 @@ def _handle_node_workspace(self, node, workspace_package, inverse_levels, deps_g
     def _build_package(self, node, package_ref, output, keep_build):
         conan_ref, conan_file = node.conan_ref, node.conanfile
 
-        skip_build = conan_file.develop and keep_build
-        if skip_build:
-            output.info("Won't be built as specified by --keep-build")
-
         t1 = time.time()
         builder = _ConanPackageBuilder(conan_file, package_ref, self._client_cache, output,
                                        self._plugin_manager)
+        if is_dirty(builder.build_folder):
+            output.warn("Build folder is dirty, removing it: %s" % builder.build_folder)
+            rmdir(builder.build_folder)
 
+        skip_build = conan_file.develop and keep_build
+        if skip_build:
+            output.info("Won't be built as specified by --keep-build")
         if skip_build:
             if not os.path.exists(builder.build_folder):
                 msg = "--keep-build specified, but build folder not found"
@@ -390,6 +392,7 @@ def _build_package(self, node, package_ref, output, keep_build):
                 raise ConanException(msg)
         else:
             with self._client_cache.conanfile_write_lock(conan_ref):
+                set_dirty(builder.build_folder)
                 complete_recipe_sources(self._remote_manager, self._client_cache,
                                         self._registry, conan_file, conan_ref)
                 builder.prepare_build()
@@ -398,6 +401,7 @@ def _build_package(self, node, package_ref, output, keep_build):
             try:
                 if not skip_build:
                     builder.build()
+                    clean_dirty(builder.build_folder)
                 builder.package()
             except ConanException as exc:
                 self._recorder.package_install_error(package_ref, INSTALL_ERROR_BUILDING,


Test patch:
diff --git a/conans/test/integration/build_id_test.py b/conans/test/integration/build_id_test.py
--- a/conans/test/integration/build_id_test.py
+++ b/conans/test/integration/build_id_test.py
@@ -265,3 +265,33 @@ def _check():
             _check()
             self.assertIn("ID: ab2e9f86b4109980930cdc685f4a320b359e7bb4", client.out)
             self.assertNotIn("ID: f3989dcba0ab50dc5ed9b40ede202bdd7b421f09", client.out)
+
+    def failed_build_test(self):
+        conanfile = """from conans import ConanFile
+class MyTest(ConanFile):
+    settings = "os"
+    def build(self):
+        raise Exception("Failed build!!")
+"""
+        client = TestClient()
+        # NORMAL case, every create fails
+        client.save({"conanfile.py": conanfile})
+        error = client.run("create . pkg/0.1@user/channel", ignore_error=True)
+        self.assertTrue(error)
+        self.assertIn("ERROR: pkg/0.1@user/channel: Error in build() method, line 5",
+                      client.out)
+        error = client.run("create . pkg/0.1@user/channel", ignore_error=True)
+        self.assertTrue(error)
+        self.assertIn("ERROR: pkg/0.1@user/channel: Error in build() method, line 5",
+                      client.out)
+        # now test with build_id
+        client.save({"conanfile.py": conanfile +
+                     "    def build_id(self): self.info_build.settings.os = 'any'"})
+        error = client.run("create . pkg/0.1@user/channel", ignore_error=True)
+        self.assertTrue(error)
+        self.assertIn("ERROR: pkg/0.1@user/channel: Error in build() method, line 5",
+                      client.out)
+        error = client.run("create . pkg/0.1@user/channel", ignore_error=True)
+        self.assertTrue(error)
+        self.assertIn("ERROR: pkg/0.1@user/channel: Error in build() method, line 5",
+                      client.out)