Repository: conan-io/conan

Issue:
Private static library dependencies
To help us debug your issue please explain:

- [✅ ] I've read the [CONTRIBUTING guide](https://raw.githubusercontent.com/conan-io/conan/develop/.github/CONTRIBUTING.md).
- [✅ ] I've specified the Conan version, operating system version and any tool that can be relevant.
- [ ✅] I've explained the steps to reproduce the error or the motivation/use case of the question/suggestion.

I am having a problem I'll try to explain with the following example:

Let's have following packages that are all static libraries: `Aaa`, `Bbb`, `Ccc`, `Ddd` and `Ooo`. Also, let `Eee` be an executable that depends on `Ddd`.

Additionally, `Ddd` depends on `Ccc`, `Ccc` depends on `Bbb` and `Bbb` depends on `Aaa`.

`Aaa` has following conanfile:

```python
class AConan(ConanFile):
    name = 'Aaa'
    version = '1.0.0'
    options = {
        'use_o': [True, False]
    }
    default_options = {
        'use_o': False
    }

    def requirements(self):
        if self.options.use_o:
            self.requires('Ooo/1.0.0@user/testing')
```

i.e. `A` has an option `use_o`, it depends on `Ooo` package. Otherwise, it does not have any dependencies.

Now, let's first build `Ooo` with `conan create` and then build both versions of `Aaa` - the version with default options and the version with `use_o=True`. This simulates what a CI server would do when building `Aaa` package.

Next, let's build packages `Bbb`, `Ccc` and `Ddd` in correct order, but using only default settings, as none of those packages care about `Aaa's` `use_o` parameter.

Now, when installing package `Eee` with `conan install` everything works OK, but if `Eee` now wants to change `Aaa:use_o` to `True` with `conan install /path/to/Eee/conanfile.py -o Aaa:use_o=True`, it will need to rebuild `Bbb`, `Ccc` and `Ddd` packages even though they do not depend at all on `Aaa's` `use_o` parameter. The `Bbb's`, `Ccc's` and `Ddd's` binary will be **exactly the same** as with `Aaa:use_o=False`, but their `package_id's` will change because there is one more dependency in the graph.

Of course, `Eee` must be aware of `Ooo` in order to correctly link to it when creating final executable, but `Bbb`, `Ccc` and `Ddd` should not be rebuilt in that case. This wastes a lot of time in our real setup, as more than 100 packages are rebuilt because the top level package changes an option in bottom level package that introduces a new dependency and this wastes a lot of time and disk space for our developers. 
All packages in between are static libraries and should not be rebuilt. Actually, sometimes they should, but that should then depend on specific option of the dependent package, as mentioned in issue #3776. Is there a way to prevent rebuilding of packages `Bbb`, `Ccc` and `Ddd` when change in `Aaa's` option introduces a new dependency to the graph, i.e. is there a way to keep `Bbb's`, `Ccc's` and `Ddd's` `package_id` unchanged after `Aaa` introduces `Ooo` as its dependency?

Here is a little demo that follows the description above:
[conan-static-deps.zip](https://github.com/conan-io/conan/files/2493207/conan-static-deps.zip)



Discussion:

Patch:
diff --git a/conans/__init__.py b/conans/__init__.py
index c5564cfc9..d6bb0a50f 100644
--- a/conans/__init__.py
+++ b/conans/__init__.py
@@ -19,4 +19,3 @@ REVISIONS = "revisions"  # Only when enabled in config, not by default look at s
 SERVER_CAPABILITIES = [COMPLEX_SEARCH_CAPABILITY, ]  # Still without v2 because it is changing
 
 __version__ = '1.9.0-dev'
-
diff --git a/conans/client/cmd/build.py b/conans/client/cmd/build.py
new file mode 100644
index 000000000..2b5f59278
--- /dev/null
+++ b/conans/client/cmd/build.py
@@ -0,0 +1,70 @@
+import os
+
+from conans.client.output import ScopedOutput
+from conans.util.log import logger
+from conans.errors import (NotFoundException, ConanException,
+                           conanfile_exception_formatter)
+from conans.paths import CONANFILE, CONANFILE_TXT
+from conans.util.files import mkdir
+from conans.model.conan_file import get_env_context_manager
+
+
+def build(graph_manager, plugin_manager, conanfile_path, output,
+          source_folder, build_folder, package_folder, install_folder,
+          test=False, should_configure=True, should_build=True, should_install=True,
+          should_test=True):
+    """ Call to build() method saved on the conanfile.py
+    param conanfile_path: path to a conanfile.py
+    """
+    logger.debug("Building in %s" % build_folder)
+    logger.debug("Conanfile in %s" % conanfile_path)
+
+    try:
+        # Append env_vars to execution environment and clear when block code ends
+        output = ScopedOutput(("%s (test package)" % test) if test else "Project",
+                              output)
+        conan_file = graph_manager.load_consumer_conanfile(conanfile_path, install_folder,
+                                                           output, deps_info_required=True)
+    except NotFoundException:
+        # TODO: Auto generate conanfile from requirements file
+        raise ConanException("'%s' file is needed for build.\n"
+                             "Create a '%s' and move manually the "
+                             "requirements and generators from '%s' file"
+                             % (CONANFILE, CONANFILE, CONANFILE_TXT))
+
+    if test:
+        try:
+            conan_file.requires.add(test)
+        except ConanException:
+            pass
+
+    conan_file.should_configure = should_configure
+    conan_file.should_build = should_build
+    conan_file.should_install = should_install
+    conan_file.should_test = should_test
+
+    try:
+        mkdir(build_folder)
+        os.chdir(build_folder)
+        conan_file.build_folder = build_folder
+        conan_file.source_folder = source_folder
+        conan_file.package_folder = package_folder
+        conan_file.install_folder = install_folder
+        plugin_manager.execute("pre_build", conanfile=conan_file,
+                               conanfile_path=conanfile_path)
+        with get_env_context_manager(conan_file):
+            output.highlight("Running build()")
+            with conanfile_exception_formatter(str(conan_file), "build"):
+                conan_file.build()
+            plugin_manager.execute("post_build", conanfile=conan_file,
+                                   conanfile_path=conanfile_path)
+            if test:
+                output.highlight("Running test()")
+                with conanfile_exception_formatter(str(conan_file), "test"):
+                    conan_file.test()
+    except ConanException:
+        raise  # Raise but not let to reach the Exception except (not print traceback)
+    except Exception:
+        import traceback
+        trace = traceback.format_exc().split('\n')
+        raise ConanException("Unable to build it successfully\n%s" % '\n'.join(trace[3:]))
diff --git a/conans/client/cmd/export_pkg.py b/conans/client/cmd/export_pkg.py
new file mode 100644
index 000000000..d5ccbe8ae
--- /dev/null
+++ b/conans/client/cmd/export_pkg.py
@@ -0,0 +1,52 @@
+import os
+
+from conans.client.output import ScopedOutput
+from conans.client import packager
+from conans.errors import ConanException
+
+from conans.util.files import rmdir
+from conans.model.ref import PackageReference
+from conans.client.graph.graph_manager import load_deps_info
+
+
+def export_pkg(client_cache, graph_manager, plugin_manager, recorder, output,
+               reference, source_folder, build_folder, package_folder, install_folder,
+               profile, force):
+
+    conan_file_path = client_cache.conanfile(reference)
+    if not os.path.exists(conan_file_path):
+        raise ConanException("Package recipe '%s' does not exist" % str(reference))
+
+    deps_graph = graph_manager.load_simple_graph(reference, profile, recorder)
+
+    # this is a bit tricky, but works. The root (virtual), has only 1 neighbor,
+    # which is the exported pkg
+    nodes = deps_graph.root.neighbors()
+    conanfile = nodes[0].conanfile
+    from conans.client.conan_api import existing_info_files
+    if install_folder and existing_info_files(install_folder):
+        load_deps_info(install_folder, conanfile, required=True)
+    pkg_id = conanfile.info.package_id()
+    output.info("Packaging to %s" % pkg_id)
+    pkg_reference = PackageReference(reference, pkg_id)
+    dest_package_folder = client_cache.package(pkg_reference,
+                                               short_paths=conanfile.short_paths)
+
+    if os.path.exists(dest_package_folder):
+        if force:
+            rmdir(dest_package_folder)
+        else:
+            raise ConanException("Package already exists. Please use --force, -f to "
+                                 "overwrite it")
+
+    recipe_hash = client_cache.load_manifest(reference).summary_hash
+    conanfile.info.recipe_hash = recipe_hash
+    conanfile.develop = True
+    package_output = ScopedOutput(str(reference), output)
+    if package_folder:
+        packager.export_pkg(conanfile, pkg_id, package_folder, dest_package_folder,
+                            package_output, plugin_manager, conan_file_path, reference)
+    else:
+        packager.create_package(conanfile, pkg_id, source_folder, build_folder,
+                                dest_package_folder, install_folder, package_output,
+                                plugin_manager, conan_file_path, reference, local=True)
diff --git a/conans/client/cmd/uploader.py b/conans/client/cmd/uploader.py
index 7cf31eebc..e81fd1a83 100644
--- a/conans/client/cmd/uploader.py
+++ b/conans/client/cmd/uploader.py
@@ -1,9 +1,10 @@
+import os
 import time
-
 from conans.client.source import complete_recipe_sources
 from conans.errors import ConanException, NotFoundException
 from conans.model.ref import PackageReference, ConanFileReference
 from conans.search.search import search_recipes, search_packages
+from conans.util.files import load
 from conans.util.log import logger
 
 
@@ -167,8 +168,31 @@ class CmdUpload(object):
 
         if (remote_recipe_manifest != local_manifest and
                 remote_recipe_manifest.time > local_manifest.time):
+            self._print_manifest_information(remote_recipe_manifest, local_manifest, conan_ref, remote)
             raise ConanException("Remote recipe is newer than local recipe: "
                                  "\n Remote date: %s\n Local date: %s" %
                                  (remote_recipe_manifest.time, local_manifest.time))
 
         return remote_recipe_manifest
+
+    def _print_manifest_information(self, remote_recipe_manifest, local_manifest, conan_ref, remote):
+        try:
+            self._user_io.out.info("\n%s" % ("-"*40))
+            self._user_io.out.info("Remote manifest:")
+            self._user_io.out.info(remote_recipe_manifest)
+            self._user_io.out.info("Local manifest:")
+            self._user_io.out.info(local_manifest)
+            difference = remote_recipe_manifest.difference(local_manifest)
+            if "conanfile.py" in difference:
+                contents = load(os.path.join(self._client_cache.export(conan_ref),
+                                                   "conanfile.py"))
+                endlines = "\\r\\n" if "\r\n" in contents else "\\n"
+                self._user_io.out.info("Local 'conanfile.py' using '%s' line-ends" % endlines)
+                remote_contents = self._remote_manager.get_path(conan_ref, package_id=None,
+                                                                path="conanfile.py", remote=remote)
+                endlines = "\\r\\n" if "\r\n" in remote_contents else "\\n"
+                self._user_io.out.info("Remote 'conanfile.py' using '%s' line-ends" % endlines)
+            self._user_io.out.info("\n%s" % ("-"*40))
+        except Exception as e:
+            self._user_io.out.info("Error printing information about the diff: %s" % str(e))
+
diff --git a/conans/client/conan_api.py b/conans/client/conan_api.py
index 1c25e4696..f068b4e64 100644
--- a/conans/client/conan_api.py
+++ b/conans/client/conan_api.py
@@ -11,9 +11,9 @@ from conans.client.plugin_manager import PluginManager
 from conans.client.recorder.action_recorder import ActionRecorder
 from conans.client.client_cache import ClientCache
 from conans.client.conf import MIN_SERVER_COMPATIBLE_VERSION, ConanClientConfigParser
-from conans.client.manager import ConanManager, existing_info_files
+from conans.client.manager import ConanManager
 from conans.client.migrations import ClientMigrator
-from conans.client.output import ConanOutput
+from conans.client.output import ConanOutput, ScopedOutput
 from conans.client.profile_loader import read_profile, profile_from_args, \
     read_conaninfo_profile
 from conans.client.recorder.search_recorder import SearchRecorder
@@ -42,8 +42,8 @@ from conans.client.cmd.profile import cmd_profile_update, cmd_profile_get,\
     cmd_profile_delete_key, cmd_profile_create, cmd_profile_list
 from conans.client.cmd.search import Search
 from conans.client.cmd.user import users_clean, users_list, user_set
-from conans.client.importer import undo_imports
-from conans.client.cmd.export import cmd_export, export_alias
+from conans.client.importer import undo_imports, run_imports
+from conans.client.cmd.export import cmd_export, export_alias, _execute_export
 from conans.unicode import get_cwd
 from conans.client.remover import ConanRemover
 from conans.client.cmd.download import download
@@ -53,6 +53,10 @@ from conans.client.loader import ConanFileLoader
 from conans.client.graph.proxy import ConanProxy
 from conans.client.graph.python_requires import ConanPythonRequire
 from conans.client.graph.range_resolver import RangeResolver
+from conans.client import packager
+from conans.client.source import config_source_local
+from conans.client.cmd.build import build
+from conans.client.cmd.export_pkg import export_pkg
 
 
 default_manifest_folder = '.conan_manifests'
@@ -398,14 +402,8 @@ class ConanAPIV1(object):
         source_folder = _make_abs_path(source_folder, cwd, default=os.path.dirname(conanfile_path))
 
         # Checks that no both settings and info files are specified
-        if install_folder and existing_info_files(install_folder) and \
-                (profile_name or settings or options or env):
-            raise ConanException("%s and %s are found, at '%s' folder, so specifying profile, "
-                                 "settings, options or env is not allowed" % (CONANINFO, BUILD_INFO,
-                                                                              install_folder))
-
         infos_present = existing_info_files(install_folder)
-        if not infos_present:
+        if profile_name or settings or options or env or not infos_present:
             profile = profile_from_args(profile_name, settings, options, env=env,
                                         cwd=cwd, client_cache=self._client_cache)
         else:
@@ -416,10 +414,11 @@ class ConanAPIV1(object):
                    self._client_cache, self._plugin_manager, self._registry)
 
         recorder = ActionRecorder()
-        manager = self._init_manager(recorder)
-        manager.export_pkg(reference, source_folder=source_folder, build_folder=build_folder,
-                           package_folder=package_folder, install_folder=install_folder,
-                           profile=profile, force=force)
+        export_pkg(self._client_cache, self._graph_manager, self._plugin_manager, recorder,
+                   self._user_io.out,
+                   reference, source_folder=source_folder, build_folder=build_folder,
+                   package_folder=package_folder, install_folder=install_folder,
+                   profile=profile, force=force)
 
     @api_method
     def download(self, reference, remote_name=None, package=None, recipe=False):
@@ -610,11 +609,10 @@ class ConanAPIV1(object):
         default_pkg_folder = os.path.join(build_folder, "package")
         package_folder = _make_abs_path(package_folder, cwd, default=default_pkg_folder)
 
-        recorder = ActionRecorder()
-        manager = self._init_manager(recorder)
-        manager.build(conanfile_path, source_folder, build_folder, package_folder,
-                      install_folder, should_configure=should_configure, should_build=should_build,
-                      should_install=should_install, should_test=should_test)
+        build(self._graph_manager, self._plugin_manager, conanfile_path, self._user_io.out,
+              source_folder, build_folder, package_folder, install_folder,
+              should_configure=should_configure, should_build=should_build,
+              should_install=should_install, should_test=should_test)
 
     @api_method
     def package(self, path, build_folder, package_folder, source_folder=None, install_folder=None,
@@ -627,10 +625,15 @@ class ConanAPIV1(object):
         default_pkg_folder = os.path.join(build_folder, "package")
         package_folder = _make_abs_path(package_folder, cwd, default=default_pkg_folder)
 
-        recorder = ActionRecorder()
-        manager = self._init_manager(recorder)
-        manager.local_package(package_folder, conanfile_path, build_folder, source_folder,
-                              install_folder)
+        if package_folder == build_folder:
+            raise ConanException("Cannot 'conan package' to the build folder. "
+                                 "--build-folder and package folder can't be the same")
+        output = ScopedOutput("PROJECT", self._user_io.out)
+        conanfile = self._graph_manager.load_consumer_conanfile(conanfile_path, install_folder,
+                                                                output, deps_info_required=True)
+        packager.create_package(conanfile, None, source_folder, build_folder, package_folder,
+                                install_folder, output, self._plugin_manager, conanfile_path, None,
+                                local=True, copy_info=True)
 
     @api_method
     def source(self, path, source_folder=None, info_folder=None, cwd=None):
@@ -643,9 +646,15 @@ class ConanAPIV1(object):
         if not os.path.exists(info_folder):
             raise ConanException("Specified info-folder doesn't exist")
 
-        recorder = ActionRecorder()
-        manager = self._init_manager(recorder)
-        manager.source(conanfile_path, source_folder, info_folder)
+        output = ScopedOutput("PROJECT", self._user_io.out)
+        # only infos if exist
+        conanfile = self._graph_manager.load_consumer_conanfile(conanfile_path, info_folder, output)
+        conanfile_folder = os.path.dirname(conanfile_path)
+        if conanfile_folder != source_folder:
+            output.info("Executing exports to: %s" % source_folder)
+            _execute_export(conanfile_path, conanfile, source_folder, source_folder, output)
+        config_source_local(source_folder, conanfile, conanfile_folder, output, conanfile_path,
+                            self._plugin_manager)
 
     @api_method
     def imports(self, path, dest=None, info_folder=None, cwd=None):
@@ -662,9 +671,10 @@ class ConanAPIV1(object):
 
         mkdir(dest)
         conanfile_abs_path = _get_conanfile_path(path, cwd, py=None)
-        recorder = ActionRecorder()
-        manager = self._init_manager(recorder)
-        manager.imports(conanfile_abs_path, dest, info_folder)
+        output = ScopedOutput("PROJECT", self._user_io.out)
+        conanfile = self._graph_manager.load_consumer_conanfile(conanfile_abs_path, info_folder,
+                                                                output, deps_info_required=True)
+        run_imports(conanfile, dest, output)
 
     @api_method
     def imports_undo(self, manifest_path):
@@ -935,6 +945,11 @@ def _parse_manifests_arguments(verify, manifests, manifests_interactive, cwd):
     return manifest_folder, manifest_interactive, manifest_verify
 
 
+def existing_info_files(folder):
+    return os.path.exists(os.path.join(folder, CONANINFO)) and  \
+           os.path.exists(os.path.join(folder, BUILD_INFO))
+
+
 def get_conan_runner():
     print_commands_to_output = get_env("CONAN_PRINT_RUN_COMMANDS", False)
     generate_run_log_file = get_env("CONAN_LOG_RUN_TO_FILE", False)
diff --git a/conans/client/manager.py b/conans/client/manager.py
index 500f6e201..75f9f3fd8 100644
--- a/conans/client/manager.py
+++ b/conans/client/manager.py
@@ -1,23 +1,19 @@
 import os
 
-from conans.client import packager
 from conans.client.client_cache import ClientCache
-from conans.client.cmd.export import _execute_export
 from conans.client.generators import write_generators
 from conans.client.importer import run_imports, run_deploy
 from conans.client.installer import ConanInstaller, call_system_requirements
 from conans.client.manifest_manager import ManifestManager
 from conans.client.output import ScopedOutput, Color
-from conans.client.source import config_source_local, complete_recipe_sources
+from conans.client.source import complete_recipe_sources
 from conans.client.tools import cross_building, get_cross_building_settings
 from conans.client.userio import UserIO
-from conans.errors import NotFoundException, ConanException, conanfile_exception_formatter
-from conans.model.conan_file import get_env_context_manager
-from conans.model.ref import ConanFileReference, PackageReference
-from conans.paths import CONANFILE, CONANINFO, CONANFILE_TXT, BUILD_INFO
-from conans.util.files import save, rmdir, normalize, mkdir
-from conans.util.log import logger
-from conans.client.graph.graph_manager import load_deps_info
+from conans.errors import ConanException
+from conans.model.ref import ConanFileReference
+from conans.paths import CONANINFO
+from conans.util.files import save, normalize
+
 from conans.client.graph.printer import print_graph
 
 
@@ -38,46 +34,6 @@ class ConanManager(object):
         self._graph_manager = graph_manager
         self._plugin_manager = plugin_manager
 
-    def export_pkg(self, reference, source_folder, build_folder, package_folder, install_folder,
-                   profile, force):
-
-        conan_file_path = self._client_cache.conanfile(reference)
-        if not os.path.exists(conan_file_path):
-            raise ConanException("Package recipe '%s' does not exist" % str(reference))
-
-        deps_graph = self._graph_manager.load_simple_graph(reference, profile, self._recorder)
-
-        # this is a bit tricky, but works. The root (virtual), has only 1 neighbor,
-        # which is the exported pkg
-        nodes = deps_graph.root.neighbors()
-        conanfile = nodes[0].conanfile
-        if install_folder and existing_info_files(install_folder):
-            load_deps_info(install_folder, conanfile, required=True)
-        pkg_id = conanfile.info.package_id()
-        self._user_io.out.info("Packaging to %s" % pkg_id)
-        pkg_reference = PackageReference(reference, pkg_id)
-        dest_package_folder = self._client_cache.package(pkg_reference,
-                                                         short_paths=conanfile.short_paths)
-
-        if os.path.exists(dest_package_folder):
-            if force:
-                rmdir(dest_package_folder)
-            else:
-                raise ConanException("Package already exists. Please use --force, -f to "
-                                     "overwrite it")
-
-        recipe_hash = self._client_cache.load_manifest(reference).summary_hash
-        conanfile.info.recipe_hash = recipe_hash
-        conanfile.develop = True
-        package_output = ScopedOutput(str(reference), self._user_io.out)
-        if package_folder:
-            packager.export_pkg(conanfile, pkg_id, package_folder, dest_package_folder,
-                                package_output, self._plugin_manager, conan_file_path, reference)
-        else:
-            packager.create_package(conanfile, pkg_id, source_folder, build_folder,
-                                    dest_package_folder, install_folder, package_output,
-                                    self._plugin_manager, conan_file_path, reference, local=True)
-
     def install_workspace(self, profile, workspace, remote_name, build_modes, update):
         references = [ConanFileReference(v, "root", "project", "develop") for v in workspace.root]
         deps_graph, _, _ = self._graph_manager.load_graph(references, None, profile, build_modes,
@@ -184,110 +140,3 @@ class ConanManager(object):
                 deploy_conanfile = neighbours[0].conanfile
                 if hasattr(deploy_conanfile, "deploy") and callable(deploy_conanfile.deploy):
                     run_deploy(deploy_conanfile, install_folder, output)
-
-    def source(self, conanfile_path, source_folder, info_folder):
-        """
-        :param conanfile_path: Absolute path to a conanfile
-        :param source_folder: Absolute path where to put the files
-        :param info_folder: Absolute path where to read the info files
-        :param package_folder: Absolute path to the package_folder, only to have the var present
-        :return:
-        """
-        output = ScopedOutput("PROJECT", self._user_io.out)
-        # only infos if exist
-        conanfile = self._graph_manager.load_consumer_conanfile(conanfile_path, info_folder, output)
-        conanfile_folder = os.path.dirname(conanfile_path)
-        if conanfile_folder != source_folder:
-            output.info("Executing exports to: %s" % source_folder)
-            _execute_export(conanfile_path, conanfile, source_folder, source_folder, output)
-        config_source_local(source_folder, conanfile, conanfile_folder, output, conanfile_path,
-                            self._plugin_manager)
-
-    def imports(self, conan_file_path, dest_folder, info_folder):
-        """
-        :param conan_file_path: Abs path to a conanfile
-        :param dest_folder:  Folder where to put the files
-        :param info_folder: Folder containing the conaninfo/conanbuildinfo.txt files
-        :return:
-        """
-
-        output = ScopedOutput("PROJECT", self._user_io.out)
-        conanfile = self._graph_manager.load_consumer_conanfile(conan_file_path, info_folder, output,
-                                                                deps_info_required=True)
-        run_imports(conanfile, dest_folder, output)
-
-    def local_package(self, package_folder, conanfile_path, build_folder, source_folder,
-                      install_folder):
-        if package_folder == build_folder:
-            raise ConanException("Cannot 'conan package' to the build folder. "
-                                 "--build-folder and package folder can't be the same")
-        output = ScopedOutput("PROJECT", self._user_io.out)
-        conanfile = self._graph_manager.load_consumer_conanfile(conanfile_path, install_folder,
-                                                                output, deps_info_required=True)
-        packager.create_package(conanfile, None, source_folder, build_folder, package_folder,
-                                install_folder, output, self._plugin_manager, conanfile_path, None,
-                                local=True, copy_info=True)
-
-    def build(self, conanfile_path, source_folder, build_folder, package_folder, install_folder,
-              test=False, should_configure=True, should_build=True, should_install=True,
-              should_test=True):
-        """ Call to build() method saved on the conanfile.py
-        param conanfile_path: path to a conanfile.py
-        """
-        logger.debug("Building in %s" % build_folder)
-        logger.debug("Conanfile in %s" % conanfile_path)
-
-        try:
-            # Append env_vars to execution environment and clear when block code ends
-            output = ScopedOutput(("%s (test package)" % test) if test else "Project",
-                                  self._user_io.out)
-            conan_file = self._graph_manager.load_consumer_conanfile(conanfile_path, install_folder,
-                                                                     output, deps_info_required=True)
-        except NotFoundException:
-            # TODO: Auto generate conanfile from requirements file
-            raise ConanException("'%s' file is needed for build.\n"
-                                 "Create a '%s' and move manually the "
-                                 "requirements and generators from '%s' file"
-                                 % (CONANFILE, CONANFILE, CONANFILE_TXT))
-
-        if test:
-            try:
-                conan_file.requires.add(test)
-            except ConanException:
-                pass
-
-        conan_file.should_configure = should_configure
-        conan_file.should_build = should_build
-        conan_file.should_install = should_install
-        conan_file.should_test = should_test
-
-        try:
-            mkdir(build_folder)
-            os.chdir(build_folder)
-            conan_file.build_folder = build_folder
-            conan_file.source_folder = source_folder
-            conan_file.package_folder = package_folder
-            conan_file.install_folder = install_folder
-            self._plugin_manager.execute("pre_build", conanfile=conan_file,
-                                         conanfile_path=conanfile_path)
-            with get_env_context_manager(conan_file):
-                output.highlight("Running build()")
-                with conanfile_exception_formatter(str(conan_file), "build"):
-                    conan_file.build()
-                self._plugin_manager.execute("post_build", conanfile=conan_file,
-                                             conanfile_path=conanfile_path)
-                if test:
-                    output.highlight("Running test()")
-                    with conanfile_exception_formatter(str(conan_file), "test"):
-                        conan_file.test()
-        except ConanException:
-            raise  # Raise but not let to reach the Exception except (not print traceback)
-        except Exception:
-            import traceback
-            trace = traceback.format_exc().split('\n')
-            raise ConanException("Unable to build it successfully\n%s" % '\n'.join(trace[3:]))
-
-
-def existing_info_files(folder):
-    return os.path.exists(os.path.join(folder, CONANINFO)) and  \
-           os.path.exists(os.path.join(folder, BUILD_INFO))
diff --git a/conans/client/packager.py b/conans/client/packager.py
index b439afd63..9b1914d7b 100644
--- a/conans/client/packager.py
+++ b/conans/client/packager.py
@@ -5,7 +5,8 @@ from conans.client import tools
 from conans.util.files import mkdir, save, rmdir
 from conans.util.log import logger
 from conans.paths import CONANINFO
-from conans.errors import ConanException, ConanExceptionInUserConanfileMethod, conanfile_exception_formatter
+from conans.errors import (ConanException, ConanExceptionInUserConanfileMethod,
+                           conanfile_exception_formatter)
 from conans.model.manifest import FileTreeManifest
 from conans.client.output import ScopedOutput
 from conans.client.file_copier import FileCopier
@@ -17,7 +18,6 @@ def export_pkg(conanfile, pkg_id, src_package_folder, package_folder, output, pl
     conanfile.package_folder = src_package_folder
     output.info("Exporting to cache existing package from user folder")
     output.info("Package folder %s" % package_folder)
-    print("export_pkg", type(reference))
     plugin_manager.execute("pre_package", conanfile=conanfile, conanfile_path=conanfile_path,
                            reference=reference, package_id=pkg_id)
 
diff --git a/conans/model/manifest.py b/conans/model/manifest.py
index dd70fc8f1..a710e8af7 100644
--- a/conans/model/manifest.py
+++ b/conans/model/manifest.py
@@ -1,10 +1,10 @@
-import os
 import calendar
+import datetime
+import os
 import time
-from conans.util.files import md5sum, md5, save, load, walk
-from conans.paths import PACKAGE_TGZ_NAME, EXPORT_TGZ_NAME, CONAN_MANIFEST, EXPORT_SOURCES_TGZ_NAME
 from conans.errors import ConanException
-import datetime
+from conans.paths import PACKAGE_TGZ_NAME, EXPORT_TGZ_NAME, CONAN_MANIFEST, EXPORT_SOURCES_TGZ_NAME
+from conans.util.files import md5sum, md5, save, load, walk
 
 
 def discarded_file(filename):
@@ -39,9 +39,9 @@ def gather_files(folder):
 
 class FileTreeManifest(object):
 
-    def __init__(self, time, file_sums):
+    def __init__(self, the_time, file_sums):
         """file_sums is a dict with filepaths and md5's: {filepath/to/file.txt: md5}"""
-        self.time = time
+        self.time = the_time
         self.file_sums = file_sums
 
     def files(self):
@@ -63,14 +63,14 @@ class FileTreeManifest(object):
         ConanDigest
         """
         tokens = text.split("\n")
-        time = int(tokens[0])
+        the_time = int(tokens[0])
         file_sums = {}
         for md5line in tokens[1:]:
             if md5line:
                 filename, file_md5 = md5line.split(": ")
                 if not discarded_file(filename):
                     file_sums[filename] = file_md5
-        return FileTreeManifest(time, file_sums)
+        return FileTreeManifest(the_time, file_sums)
 
     @staticmethod
     def load(folder):
@@ -78,16 +78,25 @@ class FileTreeManifest(object):
         return FileTreeManifest.loads(text)
 
     def __repr__(self):
-        ret = ["%s" % (self.time)]
-        for filepath, file_md5 in sorted(self.file_sums.items()):
-            ret.append("%s: %s" % (filepath, file_md5))
+        ret = ["%s" % self.time]
+        for file_path, file_md5 in sorted(self.file_sums.items()):
+            ret.append("%s: %s" % (file_path, file_md5))
+        ret.append("")
+        content = "\n".join(ret)
+        return content
+
+    def __str__(self):
+        dt = datetime.datetime.utcfromtimestamp(self.time).strftime('%Y-%m-%d %H:%M:%S')
+        ret = ["Time: %s" % dt]
+        for file_path, file_md5 in sorted(self.file_sums.items()):
+            ret.append("%s, MD5: %s" % (file_path, file_md5))
         ret.append("")
         content = "\n".join(ret)
         return content
 
     def save(self, folder, filename=CONAN_MANIFEST):
         path = os.path.join(folder, filename)
-        save(path, str(self))
+        save(path, repr(self))
 
     @classmethod
     def create(cls, folder, exports_sources_folder=None):
diff --git a/conans/requirements.txt b/conans/requirements.txt
index ce691cf17..5b4a32db0 100644
--- a/conans/requirements.txt
+++ b/conans/requirements.txt
@@ -12,4 +12,4 @@ future==0.16.0
 pygments>=2.0, <3.0
 astroid>=1.6.5
 deprecation>=2.0, <2.1
-tqdm>=4.27.0, <5.0
+tqdm-conan==4.27.0
diff --git a/conans/requirements_osx.txt b/conans/requirements_osx.txt
index b9de59ac6..39f2a5366 100644
--- a/conans/requirements_osx.txt
+++ b/conans/requirements_osx.txt
@@ -1,5 +1,3 @@
 idna==2.6 # Solving conflict, somehow is installing 2.7 when requests require 2.6
 cryptography>=1.3.4, <2.4.0
 pyOpenSSL>=16.0.0, <18.0.0
-ndg-httpsclient>=0.4.1, <0.5.0
-pyasn>=1.5.0b7, <1.6.0


Test patch:
diff --git a/conans/client/cmd/test.py b/conans/client/cmd/test.py
index 87ecc1c42..e0614c05d 100644
--- a/conans/client/cmd/test.py
+++ b/conans/client/cmd/test.py
@@ -4,6 +4,7 @@ import tempfile
 
 from conans.util.files import rmdir
 from conans.util.env_reader import get_env
+from conans.client.cmd.build import build
 
 
 class PackageTester(object):
@@ -21,7 +22,8 @@ class PackageTester(object):
         and builds the test_package/conanfile.py running the test() method.
         """
         base_folder = os.path.dirname(conanfile_abs_path)
-        test_build_folder, delete_after_build = self._build_folder(test_build_folder, profile, base_folder)
+        test_build_folder, delete_after_build = self._build_folder(test_build_folder, profile,
+                                                                   base_folder)
         rmdir(test_build_folder)
         if build_modes is None:
             build_modes = ["never"]
@@ -37,11 +39,14 @@ class PackageTester(object):
                                   manifest_verify=manifest_verify,
                                   manifest_interactive=manifest_interactive,
                                   keep_build=keep_build)
-            self._manager.build(conanfile_abs_path, base_folder, test_build_folder, package_folder=None,
-                                install_folder=test_build_folder, test=str(reference))
+            # FIXME: This is ugly access to graph_manager and plugin_manager. Will be cleaned in 2.0
+            build(self._manager._graph_manager, self._manager._plugin_manager, conanfile_abs_path,
+                  self._user_io.out,
+                  base_folder, test_build_folder, package_folder=None,
+                  install_folder=test_build_folder, test=str(reference))
         finally:
             if delete_after_build:
-                os.chdir(base_folder)    # Required for windows where deleting the cwd is not possible.
+                os.chdir(base_folder)  # Required for windows where deleting the cwd is not possible.
                 rmdir(test_build_folder)
 
     @staticmethod
diff --git a/conans/test/command/export_pkg_test.py b/conans/test/command/export_pkg_test.py
index 446272655..a073ddebb 100644
--- a/conans/test/command/export_pkg_test.py
+++ b/conans/test/command/export_pkg_test.py
@@ -29,6 +29,36 @@ class Pkg(ConanFile):
         client.run("install .")
         client.run("export-pkg . Pkg/0.1@user/testing")
 
+    def test_transitive_without_settings(self):
+        # https://github.com/conan-io/conan/issues/3367
+        conanfile = """from conans import ConanFile
+class PkgC(ConanFile):
+    pass
+"""
+        client = TestClient()
+        client.save({CONANFILE: conanfile})
+        client.run("create . PkgC/0.1@user/testing")
+        conanfile = """from conans import ConanFile
+class PkgB(ConanFile):
+    settings = "arch"
+    requires = "PkgC/0.1@user/testing"
+"""
+        client.save({CONANFILE: conanfile})
+        client.run("create . PkgB/0.1@user/testing")
+        conanfile = """from conans import ConanFile
+class PkgA(ConanFile):
+    requires = "PkgB/0.1@user/testing"
+    def build(self):
+        self.output.info("BUILDING PKGA")
+"""
+        client.save({CONANFILE: conanfile})
+        client.run("install . -if=build")
+        client.run("build . -bf=build")
+        client.run("export-pkg . PkgA/0.1@user/testing -bf=build -pr=default")
+        self.assertIn("PkgA/0.1@user/testing: Package "
+                      "'8f97510bcea8206c1c046cc8d71cc395d4146547' created",
+                      client.out)
+
     def test_package_folder_errors(self):
         # https://github.com/conan-io/conan/issues/2350
         conanfile = """from conans import ConanFile
@@ -179,45 +209,42 @@ class TestConan(ConanFile):
                       client.user_io.out)
 
         # Now repeat
-        client.save({CONANFILE: conanfile}, clean_first=True)
-        client.save({"include/header.h": "//Windows header2"})
+        client.save({CONANFILE: conanfile,
+                     "include/header.h": "//Windows header2"}, clean_first=True)
+        # Without force it fails
         err = client.run("export-pkg . Hello/0.1@lasote/stable -s os=Windows",
                          ignore_error=True)
         self.assertIn("Package already exists. Please use --force, -f to overwrite it",
                       client.user_io.out)
         self.assertTrue(err)
-
-        # Will fail because it finds the info files in the curdir.
-        client.run("install . -s os=Windows")
-        err = client.run("export-pkg . Hello/0.1@lasote/stable -s os=Windows -f", ignore_error=True)
-        self.assertTrue(err)
-        self.assertIn("conaninfo.txt and conanbuildinfo.txt are found", client.out)
-
-        client.save({CONANFILE: conanfile, "include/header.h": "//Windows header2"},
-                    clean_first=True)
+        # With force works
         client.run("export-pkg . Hello/0.1@lasote/stable -s os=Windows -f")
         self.assertEqual(load(os.path.join(package_folder, "include/header.h")),
                          "//Windows header2")
 
-        # Now use --install-folder and avoid the -s os=Windows, should fail without  -f
+        # Now use --install-folder
+        client.save({CONANFILE: conanfile,
+                     "include/header.h": "//Windows header3"}, clean_first=True)
+        # Without force it fails
         client.run("install . --install-folder=inst -s os=Windows")
         err = client.run("export-pkg . Hello/0.1@lasote/stable -if inst", ignore_error=True)
         self.assertTrue(err)
         self.assertIn("Package already exists. Please use --force, -f to overwrite it",
                       client.user_io.out)
+        # With force works
         client.run("export-pkg . Hello/0.1@lasote/stable -if inst -f")
+        self.assertIn("Hello/0.1@lasote/stable: Package '3475bd55b91ae904ac96fde0f106a136ab951a5e'"
+                      " created", client.out)
         self.assertEqual(load(os.path.join(package_folder, "include/header.h")),
-                         "//Windows header2")
-
-        # Try to specify a setting and the install folder
-        error = client.run("export-pkg . Hello/0.1@lasote/stable -if inst -s os=Linux", ignore_error=True)
-        self.assertTrue(error)
-        self.assertIn("conaninfo.txt and conanbuildinfo.txt are found", client.user_io.out)
+                         "//Windows header3")
 
-        error = client.run("export-pkg . Hello/0.1@lasote/stable -if inst -f --profile=default",
-                           ignore_error=True)
-        self.assertIn("conaninfo.txt and conanbuildinfo.txt are found", client.user_io.out)
-        self.assertTrue(error)
+        # we can specify settings too
+        client.save({"include/header.h": "//Windows header4"})
+        client.run("export-pkg . Hello/0.1@lasote/stable -if inst -f -s os=Windows")
+        self.assertIn("Hello/0.1@lasote/stable: Package '3475bd55b91ae904ac96fde0f106a136ab951a5e'"
+                      " created", client.out)
+        self.assertEqual(load(os.path.join(package_folder, "include/header.h")),
+                         "//Windows header4")
 
         # Try to specify a install folder with no files
         error = client.run("export-pkg . Hello/0.1@lasote/stable -if fake", ignore_error=True)
@@ -376,7 +403,8 @@ class TestConan(ConanFile):
         # Partial reference is ok
         client.save({CONANFILE: conanfile, "file.txt": "txt contents"})
         client.run("export-pkg . anyname/1.222@conan/stable")
-        self.assertIn("anyname/1.222@conan/stable package(): Copied 1 '.txt' file: file.txt", client.out)
+        self.assertIn("anyname/1.222@conan/stable package(): Copied 1 '.txt' file: file.txt",
+                      client.out)
 
     def test_with_deps(self):
         client = TestClient()
diff --git a/conans/test/command/upload_complete_test.py b/conans/test/command/upload_complete_test.py
index 5719cd22c..37ef391a5 100644
--- a/conans/test/command/upload_complete_test.py
+++ b/conans/test/command/upload_complete_test.py
@@ -1,19 +1,21 @@
+import os
 import json
 import unittest
+import platform
+import stat
+
+from requests.packages.urllib3.exceptions import ConnectionError
+
 from conans.test.utils.tools import TestClient, TestServer, TestRequester
 from conans.test.utils.test_files import hello_source_files, temp_folder,\
     hello_conan_files
-from conans.client.manager import CONANFILE
-import os
-from conans.paths import CONAN_MANIFEST, EXPORT_TGZ_NAME, CONANINFO
-import platform
-import stat
+from conans.paths import CONAN_MANIFEST, EXPORT_TGZ_NAME, CONANINFO, CONANFILE
 from conans.util.files import load, mkdir, save
 from conans.model.ref import ConanFileReference, PackageReference
 from conans.model.manifest import FileTreeManifest
 from conans.test.utils.test_files import uncompress_packaged_files
 from conans.tools import untargz
-from requests.packages.urllib3.exceptions import ConnectionError
+
 from conans.test.utils.cpp_test_files import cpp_hello_conan_files
 
 myconan1 = """
@@ -169,7 +171,7 @@ class UploadTest(unittest.TestCase):
         client.run("export . frodo/stable")
         client.run("upload Hello* --confirm --retry 10 --retry-wait=0", ignore_error=True)
         self.assertIn("Waiting 0 seconds to retry...", client.user_io.out)
-        self.assertIn("ERROR: Execute upload again to retry upload the failed files", client.user_io.out)
+        self.assertIn("ERROR: Execute upload again to retry upload the failed files", client.out)
 
         # For each file will fail the first time and will success in the second one
         client = self._get_client(FailPairFilesUploader)
diff --git a/conans/test/command/upload_test.py b/conans/test/command/upload_test.py
index ba978efce..aaa41a9b5 100644
--- a/conans/test/command/upload_test.py
+++ b/conans/test/command/upload_test.py
@@ -1,15 +1,14 @@
+import itertools
+import os
 import unittest
-from conans.tools import environment_append
-from conans.test.utils.tools import TestClient, TestServer
-from conans.test.utils.cpp_test_files import cpp_hello_conan_files
+from conans.errors import ConanException
 from conans.model.ref import ConanFileReference, PackageReference
+from conans.paths import EXPORT_SOURCES_TGZ_NAME, PACKAGE_TGZ_NAME
+from conans.test.utils.cpp_test_files import cpp_hello_conan_files
+from conans.test.utils.tools import TestClient, TestServer
+from conans.tools import environment_append
 from conans.util.files import save, is_dirty, gzopen_without_timestamps
-import os
-import itertools
 from mock import mock
-from conans.errors import ConanException
-from conans.paths import EXPORT_SOURCES_TGZ_NAME, PACKAGE_TGZ_NAME
-
 
 conanfile = """from conans import ConanFile
 class MyPkg(ConanFile):
@@ -200,7 +199,7 @@ class UploadTest(unittest.TestCase):
         client = self._client()
 
         client.save({"conanfile.py": conanfile,
-                     "hello.cpp": ""})
+                     "hello.cpp": "int i=0"})
         client.run("export . frodo/stable")
         client.run("upload Hello0/1.2.1@frodo/stable")
         self.assertIn("Uploading conanmanifest.txt", client.user_io.out)
@@ -208,9 +207,8 @@ class UploadTest(unittest.TestCase):
                       client.out)
 
         client2 = self._client()
-
-        client2.save({"conanfile.py": conanfile,
-                     "hello.cpp": "//comamend"})
+        client2.save({"conanfile.py": conanfile + "\r\n#end",
+                      "hello.cpp": "int i=1"})
         client2.run("export . frodo/stable")
         ref = ConanFileReference.loads("Hello0/1.2.1@frodo/stable")
         manifest = client2.client_cache.load_manifest(ref)
@@ -224,7 +222,9 @@ class UploadTest(unittest.TestCase):
         # first client tries to upload again
         error = client.run("upload Hello0/1.2.1@frodo/stable", ignore_error=True)
         self.assertTrue(error)
-        self.assertIn("ERROR: Remote recipe is newer than local recipe", client.user_io.out)
+        self.assertIn("Remote recipe is newer than local recipe", client.user_io.out)
+        self.assertIn("Local 'conanfile.py' using '\\n' line-ends", client.user_io.out)
+        self.assertIn("Remote 'conanfile.py' using '\\r\\n' line-ends", client.user_io.out)
 
     def upload_unmodified_recipe_test(self):
         client = self._client()
diff --git a/conans/test/conditional_requirements_id_test.py b/conans/test/conditional_requirements_id_test.py
new file mode 100644
index 000000000..d8ebf0e3b
--- /dev/null
+++ b/conans/test/conditional_requirements_id_test.py
@@ -0,0 +1,58 @@
+import unittest
+
+from conans.test.utils.tools import TestClient
+
+
+class ConditionalRequirementsIdTest(unittest.TestCase):
+
+    def basic_test(self):
+        # https://github.com/conan-io/conan/issues/3792
+        # Default might be improved 2.0 in https://github.com/conan-io/conan/issues/3762
+        client = TestClient()
+        conanfile = '''from conans import ConanFile
+class ConanLib(ConanFile):
+    pass
+'''
+        client.save({"conanfile.py": conanfile})
+        client.run("create . optional/0.1@user/testing")
+        conanfile = '''from conans import ConanFile
+class ConanLib(ConanFile):
+    options = {"use_lib": [True, False]}
+    default_options= "use_lib=False"
+    def requirements(self):
+        if self.options.use_lib:
+            self.requires("optional/0.1@user/testing")
+    def package_id(self):
+        if self.options.use_lib:
+            self.info.requires.remove("optional")
+'''
+        client.save({"conanfile.py": conanfile})
+        client.run("create . pkgA/0.1@user/testing")
+        self.assertIn("5ab84d6acfe1f23c4fae0ab88f26e3a396351ac9", client.out)
+        client.run("create . pkgA/0.1@user/testing -o pkgA:use_lib=True")
+        self.assertIn("b3485d1af719b8ddc636d57800186fc73cefff8d", client.out)
+        conanfile = '''from conans import ConanFile
+class ConanLib(ConanFile):
+    requires = "pkgA/0.1@user/testing"
+'''
+        client.save({"conanfile.py": conanfile})
+        client.run("create . pkgB/0.1@user/testing")
+        self.assertIn("pkgA/0.1@user/testing:5ab84d6acfe1f23c4fae0ab88f26e3a396351ac9", client.out)
+        self.assertIn("pkgB/0.1@user/testing:4778d500bd98ecb57d331d591aa43a7b4788d870", client.out)
+
+        client.save({"conanfile.py": conanfile.replace("pkgA", "pkgB")})
+        client.run("create . pkgC/0.1@user/testing")
+        self.assertIn("pkgA/0.1@user/testing:5ab84d6acfe1f23c4fae0ab88f26e3a396351ac9", client.out)
+        self.assertIn("pkgB/0.1@user/testing:4778d500bd98ecb57d331d591aa43a7b4788d870", client.out)
+        self.assertIn("pkgC/0.1@user/testing:4866ff85840783bec107794cce1bc12b7b8df188", client.out)
+
+        client.save({"conanfile.py": conanfile.replace("pkgA", "pkgC")})
+        client.run("install .")
+        self.assertIn("pkgA/0.1@user/testing:5ab84d6acfe1f23c4fae0ab88f26e3a396351ac9", client.out)
+        self.assertIn("pkgB/0.1@user/testing:4778d500bd98ecb57d331d591aa43a7b4788d870", client.out)
+        self.assertIn("pkgC/0.1@user/testing:4866ff85840783bec107794cce1bc12b7b8df188", client.out)
+
+        client.run("install . -o pkgA:use_lib=True")
+        self.assertIn("pkgA/0.1@user/testing:b3485d1af719b8ddc636d57800186fc73cefff8d", client.out)
+        self.assertIn("pkgB/0.1@user/testing:4778d500bd98ecb57d331d591aa43a7b4788d870", client.out)
+        self.assertIn("pkgC/0.1@user/testing:4866ff85840783bec107794cce1bc12b7b8df188", client.out)
diff --git a/conans/test/functional/create_package_test.py b/conans/test/functional/create_package_test.py
index 6c5e14a36..425890c6f 100644
--- a/conans/test/functional/create_package_test.py
+++ b/conans/test/functional/create_package_test.py
@@ -1,16 +1,14 @@
 import unittest
 import os
+import shutil
+
 from conans.test.utils.tools import TestClient, TestBufferConanOutput
 from conans.test.utils.test_files import hello_source_files
-from conans.client.manager import CONANFILE
 from conans.model.ref import ConanFileReference, PackageReference
-import shutil
-from conans.paths import CONANINFO
+from conans.paths import CONANINFO, CONANFILE
 from conans.client.packager import create_package
 from conans.client.loader import ConanFileLoader, ProcessedProfile
-from conans.model.settings import Settings
 from conans.client.output import ScopedOutput
-from conans.model.profile import Profile
 from conans.client.graph.python_requires import ConanPythonRequire