Repository: conan-io/conan

Issue:
[workspace] Packages within a workspace should not need to be in the workspace folder
### What is your suggestion?

Related to #18263 

## What

Using the `conan workspace add/remove` commands should allow relative paths above the current workspace folder, for instance I should be able to add a package whose folder is on the same level as my workspace by running `conan workspace add ..\some_package` or even a package whose folder is one level higher than my workspace by running `conan workspace add ..\..\some_package`.

## Why

The current design path of workspaces suggests that an intended use case is to have a git repo for a fixed workspace which includes the packages for that workspace as submodules.  This is good, I like the idea, but it doesn't cover common use cases for us, which include creating ad-hoc workspaces on a related subset of our (400+) repos, or quickly adding/removing a package to/from an existing workspace.

We already have tooling around creating virtual environments and adding git worktrees to those venvs, which also seems to be an intended use case, and in practice this works fine, but I wouldn't say it works great.  It means the venv folder becomes the workspace folder, which tends to clutter things up.  Being able to have a separate folder for the workspace within the venv would be preferable (and without having to manually edit the workspace files).

In a similar vein, having a single workspace for my day-to-day projects folder is not feasible, I currently have several (sometimes overlapping) Conan 1 workspaces for working on different areas of our codebase.  Again, I believe I can do this with the current implementation in Conan 2, but it means editing the workspace files by hand or writing scripts to do that for us instead of using the built in commands, which would be preferable.

### Have you read the CONTRIBUTING guide?

- [x] I've read the CONTRIBUTING guide


Discussion:
Thanks for the feedback @MBraedley-CN 

If I understand correctly, what you are suggesting is enabling a possible layout of the workspace file such as:

```
|- mylib1
|       |- CMakeLists.txt
|       |- ...
|- mylib2
|       |- CMakeLists.txt
|       |- ...
|- mywsfolder
|       |- conanws.py
|       |- CMakeLists.txt (the super-build)?
```

I'd like to clarify one thing about the behavior of all Conan commands. When user executes a 
```
cd mylib2
conan install .
```
Conan starts to scan the folders, going to the parent folder, looking for a ``conanws.py``/``conanws.yml`` file. If it is found, it means that the folder is in a workspace, that workspace is loaded, and it will define automatically on the fly editables such as ``mylib1/version`` and others.

If ``conan install`` cannot find the ``conanws.py`` because it is in a sibling folder, then ``conan install`` will know nothing about the current workspace, and will look for ``mylib1/version`` in the cache instead.

The alternative would be to add a ``--workspace-file=<path>`` to practically all Conan commands, but this seems a bit too non-ergonomic.
How do you envision the UX for such layout? Maybe adopting some strong convention over the ``mywsfolder`` folder name, so if we force it to be named something like ``conanws_xxx``, then it would be possible to be found? But what if you have several of them as you say in parallel, which one should be used?
  
    
That use case hasn't been a consideration for us, if we're using a workspace, then we want to be doing a `conan install` on the entire workspace, so `conan workspace install`.  If we do a `conan install` on an individual package, we know we'd be overriding any workspace settings and using packages that are either in the cache or explicitly set as editable instead of the packages that are implicitly editable through the workspace, and that we might be messing up the toolchain files for a workspace super build.

Part of the difference in our visions for how this should work probably stems from how @AneekCN and I have been using workspaces in Conan 1 over the past few years, and that we have invested a not insignificant amount of time and tooling to further improve our workflows around using workspaces.  The whole idea for us has been making it easy for us and our fellow developers to do a one button build of a set of plugins and each of the applications in our framework, and then run and debug that build.  (Note that this currently requires us to generate and copy hard-links to our plugins based on `if not self.in_local_cache`, but that is a discussion for a later time I think.)

Ultimately, we would like to use the Conan 2 version of workspaces for the same purpose, even if it means we still have to add some tooling around it in order to create a CMakeLists.txt (or perhaps a cn_ws.cmake file) that can be used to open/edit/build the entire workspace within Visual Studio.

So, the workflow for a fresh workspace might be as follows:

```Powershell
conan workspace create my_ws  # create an empty workspace
cd my_ws
conan workspace add --packages liba, libb --product app1  # add packages to the workspace, ./ is assumed
conan workspace install  # performs a conan install for all workspace packages
                         # and potentially generates CMakeLists.txt for a super build
cmake --preset default  # optional for IDEs that have a CMake integration like Visual Studio
```

at which point editing, compiling, and debugging the entire workspace would be possible.
Thanks very much for the feedback.

I think it can be a reasonable trade-off, as long as users understand that such a layout will not automatically activate the workspace unless they move to the folder or reference the folder, it will be good.

Lets try to make "sibling" workspace definitions possible.



Patch:
diff --git a/conan/api/subapi/workspace.py b/conan/api/subapi/workspace.py
index 10d04f77463..70b99f37919 100644
--- a/conan/api/subapi/workspace.py
+++ b/conan/api/subapi/workspace.py
@@ -1,6 +1,7 @@
 import inspect
 import os
 import shutil
+import textwrap
 from pathlib import Path
 
 from conan import ConanFile
@@ -9,7 +10,7 @@
 from conan.cli import make_abs_path
 from conan.errors import ConanException
 from conan.internal.conan_app import ConanApp
-from conan.internal.model.workspace import Workspace
+from conan.internal.model.workspace import Workspace, WORKSPACE_YML, WORKSPACE_PY
 from conan.tools.scm import Git
 from conan.internal.graph.graph import RECIPE_EDITABLE, DepsGraph, CONTEXT_HOST, RECIPE_VIRTUAL, Node, \
     RECIPE_CONSUMER
@@ -17,13 +18,13 @@
 from conan.internal.graph.profile_node_definer import consumer_definer
 from conan.internal.loader import load_python_file
 from conan.internal.source import retrieve_exports_sources
-from conan.internal.util.files import merge_directories
+from conan.internal.util.files import merge_directories, save
 
 
 def _find_ws_folder():
     path = Path(os.getcwd())
     while path.is_dir() and len(path.parts) > 1:  # finish at '/'
-        if (path / "conanws.yml").is_file() or (path / "conanws.py").is_file():
+        if (path / WORKSPACE_YML).is_file() or (path / WORKSPACE_PY).is_file():
             return str(path)
         else:
             path = path.parent
@@ -32,10 +33,10 @@ def _find_ws_folder():
 def _load_workspace(ws_folder, conan_api):
     """ loads a conanfile basic object without evaluating anything, returns the module too
     """
-    wspy = os.path.join(ws_folder, "conanws.py")
+    wspy = os.path.join(ws_folder, WORKSPACE_PY)
     if not os.path.isfile(wspy):
-        ConanOutput().info(f"conanws.py doesn't exist in {ws_folder}, using default behavior")
-        assert os.path.exists(os.path.join(ws_folder, "conanws.yml"))
+        ConanOutput().info(f"{WORKSPACE_PY} doesn't exist in {ws_folder}, using default behavior")
+        assert os.path.exists(os.path.join(ws_folder, WORKSPACE_YML))
         ws = Workspace(ws_folder, conan_api)
     else:
         try:
@@ -43,7 +44,7 @@ def _load_workspace(ws_folder, conan_api):
             ws = _parse_module(module, module_id)
             ws = ws(ws_folder, conan_api)
         except ConanException as e:
-            raise ConanException(f"Error loading conanws.py at '{wspy}': {e}")
+            raise ConanException(f"Error loading {WORKSPACE_PY} at '{wspy}': {e}")
     return ws
 
 
@@ -170,8 +171,8 @@ def open(self, require, remotes, cwd=None):
 
     def _check_ws(self):
         if not self._folder:
-            raise ConanException("Workspace not defined, please create a "
-                                 "'conanws.py' or 'conanws.yml' file")
+            raise ConanException(f"Workspace not defined, please create a "
+                                 f"'{WORKSPACE_PY}' or '{WORKSPACE_YML}' file")
 
     def add(self, path, name=None, version=None, user=None, channel=None, cwd=None,
             output_folder=None, remotes=None, product=False):
@@ -201,6 +202,30 @@ def add(self, path, name=None, version=None, user=None, channel=None, cwd=None,
         self._ws.add(ref, full_path, output_folder, product)
         return ref
 
+    @staticmethod
+    def init(path):
+        abs_path = make_abs_path(path)
+        os.makedirs(abs_path, exist_ok=True)
+        ws_yml_file = Path(abs_path, WORKSPACE_YML)
+        ws_py_file = Path(abs_path, WORKSPACE_PY)
+        if not ws_yml_file.exists():
+            ConanOutput().success(f"Created empty {WORKSPACE_YML} in {path}")
+            save(ws_yml_file, "")
+        if not ws_py_file.exists():
+            ConanOutput().success(f"Created minimal {WORKSPACE_PY} in {path}")
+            ws_name = os.path.basename(abs_path)
+            save(ws_py_file, textwrap.dedent(f'''\
+            from conan import Workspace
+
+            class MyWorkspace(Workspace):
+               """
+               Minimal Workspace class definition.
+               More info: https://docs.conan.io/2/incubating.html#workspaces
+               """
+               def name(self):
+                  return "{ws_name}"
+            '''))
+
     def remove(self, path):
         self._check_ws()
         return self._ws.remove(path)
@@ -223,11 +248,11 @@ def editable_from_path(self, path):
                 return RecipeReference.loads(ref)
 
     def collapse_editables(self, deps_graph, profile_host, profile_build):
-        ConanOutput().title(f"Collapsing workspace editables")
+        ConanOutput().title("Collapsing workspace editables")
 
         root_class = self._ws.root_conanfile()
         if root_class is not None:
-            conanfile = root_class("conanws.py base project Conanfile")
+            conanfile = root_class(f"{WORKSPACE_PY} base project Conanfile")
             consumer_definer(conanfile, profile_host, profile_build)
             root = Node(None, conanfile, context=CONTEXT_HOST, recipe=RECIPE_CONSUMER,
                         path=self._folder)  # path lets use the conanws.py folder
@@ -237,7 +262,7 @@ def collapse_editables(self, deps_graph, profile_host, profile_build):
                 if getattr(conanfile, field, None):
                     raise ConanException(f"Conanfile in conanws.py shouldn't have '{field}'")
         else:
-            ConanOutput().info("Workspace conanfilews.py not found in the workspace folder, "
+            ConanOutput().info(f"Workspace {WORKSPACE_PY} not found in the workspace folder, "
                                "using default behavior")
             conanfile = ConanFile(display_name="cli")
             consumer_definer(conanfile, profile_host, profile_build)
diff --git a/conan/cli/commands/workspace.py b/conan/cli/commands/workspace.py
index 871ae2bcb00..f1ea664e132 100644
--- a/conan/cli/commands/workspace.py
+++ b/conan/cli/commands/workspace.py
@@ -225,6 +225,18 @@ def workspace_clean(conan_api: ConanAPI, parser, subparser, *args):
     conan_api.workspace.clean()
 
 
+@conan_subcommand()
+def workspace_init(conan_api: ConanAPI, parser, subparser, *args):
+    """
+    Clean the temporary build folders when possible
+    """
+    subparser.add_argument("path", nargs="?", default=os.getcwd(),
+                        help="Path to a folder where the workspace will be initialized. "
+                             "If  does not exist")
+    args = parser.parse_args(*args)
+    conan_api.workspace.init(args.path)
+
+
 @conan_command(group="Consumer")
 def workspace(conan_api, parser, *args):  # noqa
     """
diff --git a/conan/internal/model/workspace.py b/conan/internal/model/workspace.py
index ec3b096bc24..f41ca81454a 100644
--- a/conan/internal/model/workspace.py
+++ b/conan/internal/model/workspace.py
@@ -7,6 +7,10 @@
 from conan.errors import ConanException
 from conan.internal.util.files import load, save
 
+# Related files
+WORKSPACE_YML = "conanws.yml"
+WORKSPACE_PY = "conanws.py"
+
 
 class Workspace:
     """
@@ -23,13 +27,13 @@ def name(self):
         return self.conan_data.get("name") or os.path.basename(self.folder)
 
     def _conan_load_data(self):
-        data_path = os.path.join(self.folder, "conanws.yml")
+        data_path = os.path.join(self.folder, WORKSPACE_YML)
         if not os.path.exists(data_path):
             return {}
         try:
             data = yaml.safe_load(load(data_path))
         except Exception as e:
-            raise ConanException("Invalid yml format at {}: {}".format("conanws.yml", e))
+            raise ConanException("Invalid yml format at {}: {}".format(WORKSPACE_YML, e))
         return data or {}
 
     def home_folder(self):
@@ -44,7 +48,7 @@ def add(self, ref, path, output_folder, product=False):
         self.conan_data.setdefault("editables", {})[str(ref)] = editable
         if product:
             self.conan_data.setdefault("products", []).append(path)
-        save(os.path.join(self.folder, "conanws.yml"), yaml.dump(self.conan_data))
+        save(os.path.join(self.folder, WORKSPACE_YML), yaml.dump(self.conan_data))
 
     def remove(self, path):
         found_ref = None
@@ -58,7 +62,7 @@ def remove(self, path):
         self.conan_data["editables"].pop(found_ref)
         if path in self.conan_data.get("products", []):
             self.conan_data["products"].remove(path)
-        save(os.path.join(self.folder, "conanws.yml"), yaml.dump(self.conan_data))
+        save(os.path.join(self.folder, WORKSPACE_YML), yaml.dump(self.conan_data))
         return found_ref
 
     def clean(self):
@@ -80,9 +84,6 @@ def _conan_rel_path(self, path):
         if not os.path.isabs(path):
             raise ConanException(f"Editable path must be absolute: {path}")
         path = os.path.relpath(path, self.folder)
-        if path.startswith(".."):
-            raise ConanException(f"Editable path must be inside the workspace folder: "
-                                 f"{self.folder}")
         return path.replace("\\", "/")  # Normalize to unix path
 
     def editables(self):


Test patch:
diff --git a/test/functional/workspace/test_workspace.py b/test/functional/workspace/test_workspace.py
index 78978ebafb0..4fc8eb58097 100644
--- a/test/functional/workspace/test_workspace.py
+++ b/test/functional/workspace/test_workspace.py
@@ -1,12 +1,13 @@
 import os
 import platform
+import textwrap
 
 import pytest
 
 from conan.api.subapi.workspace import WorkspaceAPI
+from conan.internal.util.files import chdir
 from conan.test.utils.tools import TestClient
 
-
 WorkspaceAPI.TEST_ENABLED = "will_break_next"
 
 
@@ -47,3 +48,49 @@ def test_metabuild():
     assert "Adding project app1" in c.out
     c.run_command("cmake --build --preset conan-release")
     # it doesn't fail
+
+
+@pytest.mark.tool("cmake")
+def test_relative_paths():
+    # This is using the meta-project
+    c = TestClient()
+    c.run("new cmake_lib -d name=liba --output=liba")
+    c.run("new cmake_exe -d name=app1 -d requires=liba/0.1 --output=app1")
+    c.run("new cmake_lib -d name=libb --output=other/libb")
+    c.run("new cmake_exe -d name=app2 -d requires=libb/0.1 --output=other/app2")
+    c.run("workspace init mywks")
+    c.run("workspace init otherwks")
+    # cd mywks
+    with c.chdir("mywks"):
+        c.run("workspace add ../liba")
+        c.run("workspace add ../app1 --product")
+        c.run("workspace build")
+        conanws_yml = textwrap.dedent("""\
+        editables:
+          app1/0.1:
+            path: ../app1
+          liba/0.1:
+            path: ../liba
+        products:
+        - ../app1
+        """)
+        assert conanws_yml == c.load("conanws.yml")
+    # cd otherwks
+    with c.chdir("otherwks"):
+        c.run("workspace add ../other/libb")
+        c.run("workspace add ../other/app2 --product")
+        c.run("workspace build")
+        conanws_yml = textwrap.dedent("""\
+        editables:
+          app2/0.1:
+            path: ../other/app2
+          libb/0.1:
+            path: ../other/libb
+        products:
+        - ../other/app2
+        """)
+        assert conanws_yml == c.load("conanws.yml")
+
+    ext = ".exe" if platform.system() == "Windows" else ""
+    assert os.path.exists(os.path.join(c.current_folder, "app1", "build", "Release", f"app1{ext}"))
+    assert os.path.exists(os.path.join(c.current_folder, "other", "app2", "build", "Release", f"app2{ext}"))
diff --git a/test/integration/workspace/test_workspace.py b/test/integration/workspace/test_workspace.py
index 19095daa709..d364fc5efc4 100644
--- a/test/integration/workspace/test_workspace.py
+++ b/test/integration/workspace/test_workspace.py
@@ -484,7 +484,7 @@ def test_install(self):
         c.run("workspace add liba")
         c.run("workspace add libb")
         c.run("workspace install -g CMakeDeps -g CMakeToolchain -of=build --envs-generation=false")
-        assert "Workspace conanfilews.py not found in the workspace folder, using default" in c.out
+        assert "Workspace conanws.py not found in the workspace folder, using default" in c.out
         files = os.listdir(os.path.join(c.current_folder, "build"))
         assert "conan_toolchain.cmake" in files
         assert "dep1-config.cmake" in files