Repository: conan-io/conan
Pull request: proposal for deploy() feature
Changelog: Feature: New ``deploy()`` method in recipes for explicit per-recipe deployment.
Docs: https://github.com/conan-io/docs/pull/3494

Close https://github.com/conan-io/conan/issues/14811

Problem statement:
[feature] Add a `deploy()` method to conanfile for application specific deployment logic
### What is your suggestion?

Hi Team,

# Bottom Line Up Front
### The proposal

Add a `deploy()` method to the conanfile, that would allow the author to specify very specific deployment instructions for applications where a more general deployment script is not appropriate. This might be invoked as follows:

```python
$ conan install --requires="myapp/1.0.0" --deploy /path/to/deployment/location
```
This does not necessarily need to conflict with `--deployer` (both may be useful for different applications, i.e. a custom deployer to aggregate licenses or generate a SBOM is not mutually exclusive with deployment for production usage).

### The use case
It would be good to have a method of writing a deployment mechanism for a single application. My specific use case here is for a GUI application that is unlikely to be invoked via the command line or with a virtual environment activated. Without knowing a heap about these applications, it might be useful to think of how an application such as Adobe Photoshop or Blender might be managed and deployed using conan.

For this case, these applications have the following properties:
* Highly specific project structure that needs to aggregate dependencies and store/read configuration from the deployed location
* Not useful as part of any developer process - this is an end-point deployment
* Unlikely to be executed from the command line
* Unlikely to use virtual environments (even though these are useful, some legacy applications won't play nicely)

The current method of doing this might be to use option (3) below, however it seems to me that this is at the wrong level of abstraction. If I have a number of applications like this, each would require their own deployer to live in the cache, which would necessitate distribution of every applications deployment recipe with global configuration. There is a UX issue here that could lead to one of these highly specific custom deployers being used to deploy packages they're not intended for, and would fail and throw a bunch of errors. It seems to me that for highly application specific deployment requirements, this deploy script/recipe is best placed with the conanfile of the applications being deployed itself.

# Context - A summary of alternatives

Currently, there are a few of methods that we have available for application deployment:

### 1. Don't deploy

For many use cases, especially when using build tools or other command line utilities, this may be sufficient:

```bash
conan install --tool_requires="cmake/[>3.25]"
conanbuild.bat
cmake.exe --version
# when you are done
deactivate_conanbuild.bat
``` 

And this works great when the application can be engaged from the command line. This does not satisfy the use case of system level deployment of a GUI application that expects to be bundled with its dependencies somewhere.

### 2. Built in deployers
Otherwise, for deployment, the build in deployers are great for simple applications or applications that adhere more to the unix philosophy in terms of application structure

```bash
$ conan install --requires="cmake/[>3.25]" --deployer=direct_deploy
$ tree direct_deploy -L 3
direct_deploy
└── cmake
    ├── bin
    │   ├── ccmake
    │   ├── cmake
    │   ├── cmake-gui
    │   ├── cpack
    │   └── ctest
    ├── conanbuildenv-x86_64.sh
    ├── conanbuild.sh
    ├── conaninfo.txt
    ├── conanmanifest.txt
    ├── conanrunenv-x86_64.sh
    ├── conanrun.sh
    ├── deactivate_conanbuild.sh
    ├── deactivate_conanrun.sh
    ├── licenses
    │   └── Copyright.txt
    └── share
        ├── aclocal
        ├── applications
        ├── bash-completion
        ├── cmake-3.25
        ├── emacs
        ├── icons
        ├── mime
        └── vim
```
You can invoke the generated environment files and use them temporarily, or you can add them to `PATH` in a more permanent way if desired. This may work excellently on unix like systems where these structures can be injected into the system easily.

However, it's not flexible enough to satisfy the use case of system level deployment of a GUI application that can't be easily invoked by command line, or where the deployment is targeting users that are not comfortable with the command line.

### 3. Custom global deployer

These are also great when built in deployers don't do what we need them to, and shine especially in situations where the deployer can perform a deployment that applies to a large number of packages. I.e., if this deploys `cmake` in a way that would be equally as valid as deploying `meson` or `protobuf`.

```bash
$ conan install --requires="cmake/[>3.25]" --deployer=mycustomdeployer
$ conan install --requires="meson/[>1.0.0]" --deployer=mycustomdeployer # Great! adheres to DRY principle and we get re-use of common deployment logic
```

This _does_ satisfy the use case - I _can_ write a highly specific custom deployer for my application here. The objection I have to this method is that it lives within the `$CONAN_HOME/extensions/deploy` directory and can be applied to other applications where it would not make sense. To illustrate, I could:

```bash
$ conan install --requires="myapp/[>3.25]" --deployer=myapp # This would work great
```
Which would work, but I could also:

```bash
$ conan install --requires="myotherapp/[>3.25]" --deployer=myapp # Error! This makes no sense
```

If I need to write a deployer that is highly specific to a single application, I think that there should be a way of coupling this with the package recipe rather than having to distribute multiple components (the deployer and the recipe) separately to make this possible.

### Have you read the CONTRIBUTING guide?

- [X] I've read the CONTRIBUTING guide

Hints from issue discussion:
This would be useful for our use cases too. 

Currently when we create a product installer, it pulls down Conan dependencies, copies files while changing the layout slightly, add a few installation scripts, then create/publish a mega-package. It's a pretty big waste of storage and network throughput.

@memsharded mentioned that what we could have done instead, is to create a conanfile.py with a `requirements`, `generate`, and `build` function, publish only the recipe, then rely on `conan install --build=missing` . While this gets us mostly there, I wonder if we would be abusing some Conan features, especially since we want to call some installation scripts. @samuel-emrys's proposal seems like a better fit. 

We were also exploring using Conan as a way to distribute and install common tools to developers which may not be related to the C++ build. We'd prefer developers to stop installing random tools from the public internet, and only the ones we have vetted. The `deploy()` function seems useful here too.

Note that we only use our private conan server and don't rely on cci at all. 

Hi @memsharded, is this something that could be considered?
Hi @samuel-emrys 

Yes, sorry for the delay.
Sure, this is something that could be considered. I can't guarantee anything, but I'll take it with the team and discuss about it. Thanks for the suggestion!
This would be very early stage, but if you want to give it a try for early feedback, very welcome: https://github.com/conan-io/conan/pull/15172

Use ``--deployer-pkg=*`` to specify the packages that we want to deploy. I think it is a valid use case that there could be more packages with deployment instructions, and having a explicit argument to specify which one or which ones have to be deployed makes sense.

Commit message: Merge branch 'release/2.0' into feature/deploy

Patch to review:
diff --git a/conan/api/subapi/install.py b/conan/api/subapi/install.py
index 5558b1de907..7fa09cebec3 100644
--- a/conan/api/subapi/install.py
+++ b/conan/api/subapi/install.py
@@ -40,7 +40,7 @@ def install_sources(self, graph, remotes):
 
     # TODO: Look for a better name
     def install_consumer(self, deps_graph, generators=None, source_folder=None, output_folder=None,
-                         deploy=False, deploy_folder=None):
+                         deploy=False, deploy_package=None, deploy_folder=None):
         """ Once a dependency graph has been installed, there are things to be done, like invoking
         generators for the root consumer.
         This is necessary for example for conanfile.txt/py, or for "conan install <ref> -g
@@ -61,9 +61,9 @@ def install_consumer(self, deps_graph, generators=None, source_folder=None, outp
         conanfile.folders.set_base_folders(source_folder, output_folder)
 
         # The previous .set_base_folders has already decided between the source_folder and output
-        if deploy:
+        if deploy or deploy_package:
             base_folder = deploy_folder or conanfile.folders.base_build
-            do_deploys(self.conan_api, deps_graph, deploy, base_folder)
+            do_deploys(self.conan_api, deps_graph, deploy, deploy_package, base_folder)
 
         conanfile.generators = list(set(conanfile.generators).union(generators or []))
         app = ConanApp(self.conan_api.cache_folder, self.conan_api.config.global_conf)
diff --git a/conan/cli/commands/graph.py b/conan/cli/commands/graph.py
index fd6de67271f..3372feb29b2 100644
--- a/conan/cli/commands/graph.py
+++ b/conan/cli/commands/graph.py
@@ -173,7 +173,7 @@ def graph_info(conan_api, parser, subparser, *args):
         conan_api.lockfile.save_lockfile(lockfile, args.lockfile_out, cwd)
         if args.deployer:
             base_folder = args.deployer_folder or os.getcwd()
-            do_deploys(conan_api, deps_graph, args.deployer, base_folder)
+            do_deploys(conan_api, deps_graph, args.deployer, None, base_folder)
 
     return {"graph": deps_graph,
             "field_filter": args.filter,
diff --git a/conan/cli/commands/install.py b/conan/cli/commands/install.py
index 16ddfb0bc35..3e17ea1dcdc 100644
--- a/conan/cli/commands/install.py
+++ b/conan/cli/commands/install.py
@@ -33,6 +33,8 @@ def install(conan_api, parser, *args):
                         help='Deploy using the provided deployer to the output folder')
     parser.add_argument("--deployer-folder",
                         help="Deployer output folder, base build folder by default if not set")
+    parser.add_argument("--deploy-package", action="append",
+                        help="Execute the deployment of the specific packages")
     parser.add_argument("--build-require", action='store_true', default=False,
                         help='Whether the provided path is a build-require')
     args = parser.parse_args(*args)
@@ -70,7 +72,8 @@ def install(conan_api, parser, *args):
     conan_api.install.install_binaries(deps_graph=deps_graph, remotes=remotes)
     ConanOutput().title("Finalizing install (deploy, generators)")
     conan_api.install.install_consumer(deps_graph, args.generator, source_folder, output_folder,
-                                       deploy=args.deployer, deploy_folder=args.deployer_folder)
+                                       deploy=args.deployer, deploy_package=args.deploy_package,
+                                       deploy_folder=args.deployer_folder)
     ConanOutput().success("Install finished successfully")
 
     # Update lockfile if necessary
diff --git a/conan/internal/deploy.py b/conan/internal/deploy.py
index 6edbba3e88a..48d58c7f257 100644
--- a/conan/internal/deploy.py
+++ b/conan/internal/deploy.py
@@ -5,6 +5,7 @@
 from conan.api.output import ConanOutput
 from conans.client.loader import load_python_file
 from conans.errors import ConanException
+from conans.model.recipe_ref import ref_matches
 from conans.util.files import rmdir, mkdir
 
 
@@ -37,8 +38,19 @@ def _load(path):
     raise ConanException(f"Cannot find deployer '{d}'")
 
 
-def do_deploys(conan_api, graph, deploy, deploy_folder):
+def do_deploys(conan_api, graph, deploy, deploy_package, deploy_folder):
     mkdir(deploy_folder)
+    # handle the recipe deploy()
+    if deploy_package:
+        for node in graph.ordered_iterate():
+            conanfile = node.conanfile
+            if not conanfile.ref or not any(ref_matches(conanfile.ref, p, None)
+                                            for p in deploy_package):
+                continue
+            if hasattr(conanfile, "deploy"):
+                conanfile.output.info("Executing deploy()")
+                conanfile.deploy_folder = deploy_folder
+                conanfile.deploy()
     # Handle the deploys
     cache = HomePaths(conan_api.cache_folder)
     for d in deploy or []:
diff --git a/conans/test/integration/conanfile/test_deploy_method.py b/conans/test/integration/conanfile/test_deploy_method.py
new file mode 100644
index 00000000000..1f2d77f5476
--- /dev/null
+++ b/conans/test/integration/conanfile/test_deploy_method.py
@@ -0,0 +1,46 @@
+import textwrap
+
+from conans.test.utils.tools import TestClient
+
+
+def test_deploy_method():
+    c = TestClient()
+    conanfile = textwrap.dedent("""
+        import os
+        from conan import ConanFile
+        from conan.tools.files import copy, save
+        class Pkg(ConanFile):
+            name = "{name}"
+            version = "0.1"
+            {requires}
+            def package(self):
+                save(self, os.path.join(self.package_folder, f"my{name}file.txt"), "HELLO!!!!")
+            def deploy(self):
+                copy(self, "*", src=self.package_folder, dst=self.deploy_folder)
+            """)
+    c.save({"dep/conanfile.py": conanfile.format(name="dep", requires=""),
+            "pkg/conanfile.py": conanfile.format(name="pkg", requires="requires='dep/0.1'")})
+    c.run("create dep")
+    assert "Executing deploy()" not in c.out
+    c.run("create pkg")
+    assert "Executing deploy()" not in c.out
+
+    # Doesn't install by default
+    c.run("install --requires=pkg/0.1")
+    assert "Executing deploy()" not in c.out
+
+    # Doesn't install with other patterns
+    c.run("install --requires=pkg/0.1 --deploy-package=other")
+    assert "Executing deploy()" not in c.out
+
+    # install can deploy all
+    c.run("install --requires=pkg/0.1 --deploy-package=* --deployer-folder=mydeploy")
+    assert "dep/0.1: Executing deploy()" in c.out
+    assert "pkg/0.1: Executing deploy()" in c.out
+    assert c.load("mydeploy/mydepfile.txt") == "HELLO!!!!"
+    assert c.load("mydeploy/mypkgfile.txt") == "HELLO!!!!"
+
+    # install can deploy only "pkg"
+    c.run("install --requires=pkg/0.1 --deploy-package=pkg/* --deployer-folder=mydeploy")
+    assert "dep/0.1: Executing deploy()" not in c.out
+    assert "pkg/0.1: Executing deploy()" in c.out


Code review comments:
diff_hunk: @@ -33,6 +33,8 @@ def install(conan_api, parser, *args):
                         help='Deploy using the provided deployer to the output folder')
     parser.add_argument("--deployer-folder",
                         help="Deployer output folder, base build folder by default if not set")
+    parser.add_argument("--deploy-package", action="append",
path: conan/cli/commands/install.py
text: @user1:
While I understand that `--deploy-package` gives a better meaning of performing an action, I think that having symmetric names with the two options above feels cleaner? My suggestion would thus be `--deployer-package`

@user2:
Yes, I think that's a valid point, not having to remember if it was _deploy_ or _deployer_ everytime so for me it's fine `--deployer-package`

@author:
Sure, lets change it

@author:
Done