Repository: conan-io/conan

Issue:
[question] profile search order
From some testing, it seems conan tries to resolve relative paths to profiles in the following order:

1. resolve from `.conan/profiles`
2. resolve from current directory

I'm a bit confused why this order was chosen, as it doesn't seem to be intuitive: Why try the more general solution before the more specific one?

---

I stumbled upon this when developing some profiles to later be shared and used with `conan config install`. One referenced profile from the local directory structure (path scheme: `../profiles/xyz`) caused some confusion, as there was another profile installed at `.conan/profiles/xyz` and the latter was chosen instead.
[question] profile search order
From some testing, it seems conan tries to resolve relative paths to profiles in the following order:

1. resolve from `.conan/profiles`
2. resolve from current directory

I'm a bit confused why this order was chosen, as it doesn't seem to be intuitive: Why try the more general solution before the more specific one?

---

I stumbled upon this when developing some profiles to later be shared and used with `conan config install`. One referenced profile from the local directory structure (path scheme: `../profiles/xyz`) caused some confusion, as there was another profile installed at `.conan/profiles/xyz` and the latter was chosen instead.


Discussion:
Hi @PengolodhGoedath 

It's not clear which steps did you use to validate it. Could you please share your steps?

Indeed Conan tries first `.conan/profiles` because it's the default path. As documented, [profiles](https://docs.conan.io/en/latest/reference/profiles.html) can be used as absolute path, and that's the recommended way. You still can set `PROFILE_DIR` env var, if you don't want use default profile folder.

Unfortunately, even if we find it wrong, we could not change that behavior, due Conan 1.0 backward compatibility, but we can think about for Conan 2.0.
@uilianries For full context, I've had the following folder structure in a repo:

```
./
    .git/
    system/
        llvm/
             conanfile.py
    profiles/
        msvc2019
```
(I skipped irrelevant parts for demonstration purposes. Yes, I'm using MSVC to build LLVM.) Now, with a terminal in `./system`, I tried to build `conan create llvm -pr ../profiles/msvc2019 -s build_type=Release`. That build failed, because it didn't actually take the profile from `./profiles/msvc2019`, and instead took a profile from `.conan/profiles/msvc2019`. (I mostly noticed because some `build_requires` listed in `./profiles/msvc2019` were missing.)

I expected the relative path to search from the local directory first, before looking in the global one (if at all). To be fair, the documentation you linked only handles relative paths starting with `.`, not `..`, but it really surprised me. I expected all relative paths to be handled equally.

Even worse: After renaming `.conan/profiles/msvc2019`, the command above worked. So it's not like conan is completely unable to handle relative paths starting with `..`, it just prioritizes questionably and looks in `PROFILE_DIR` first.

After some testing, I also found that `include()` inside of a profile has the same behavior.

---

If feel this is a bad user experience. It makes for very brittle automation scripts (e.g. automatically building system library packages, what I was trying), where they might work in one environment, but fail in another. It also goes against common intuition, expecting all relative paths to work alike. It also goes against common practice of looking in more specific (= more local) places first before looking in more general ones.

---

I'm mostly asking whether this was intended, and hopefully an explanation why it was decided this way. If it isn't intended, I might make this into a bug report, but for now I'm just assuming I'm not getting the full picture.
> Now, with a terminal in ./system, I tried to build conan create llvm -pr ../profiles/msvc2019 -s build_type=Release. That build failed, because it didn't actually take the profile from ./profiles/msvc2019, and instead took a profile from .conan/profiles/msvc2019. (I mostly noticed because some build_requires listed in ./profiles/msvc2019 were missing.)

Indeed I can reproduce your behavior running Conan 1.34.1.

It works for absolute path, but as the documentation says:

> Profiles can be located in different folders. For example, the default <userhome>/.conan/profiles, and be referenced by absolute or relative path:

> $ conan install . --profile /abs/path/to/profile  # abs path
> $ conan install . --profile ./relpath/to/profile  # resolved to current dir
> $ conan install . --profile profile  # resolved to user/.conan/profiles/profile

It doesn't mention ../ but, oblivious, it should work too, because ./msvc2019 works. I would consider it a bug. I'll investigate better. Thanks for reporting your case.


Hi @PengolodhGoedath 

It's not clear which steps did you use to validate it. Could you please share your steps?

Indeed Conan tries first `.conan/profiles` because it's the default path. As documented, [profiles](https://docs.conan.io/en/latest/reference/profiles.html) can be used as absolute path, and that's the recommended way. You still can set `PROFILE_DIR` env var, if you don't want use default profile folder.

Unfortunately, even if we find it wrong, we could not change that behavior, due Conan 1.0 backward compatibility, but we can think about for Conan 2.0.
@uilianries For full context, I've had the following folder structure in a repo:

```
./
    .git/
    system/
        llvm/
             conanfile.py
    profiles/
        msvc2019
```
(I skipped irrelevant parts for demonstration purposes. Yes, I'm using MSVC to build LLVM.) Now, with a terminal in `./system`, I tried to build `conan create llvm -pr ../profiles/msvc2019 -s build_type=Release`. That build failed, because it didn't actually take the profile from `./profiles/msvc2019`, and instead took a profile from `.conan/profiles/msvc2019`. (I mostly noticed because some `build_requires` listed in `./profiles/msvc2019` were missing.)

I expected the relative path to search from the local directory first, before looking in the global one (if at all). To be fair, the documentation you linked only handles relative paths starting with `.`, not `..`, but it really surprised me. I expected all relative paths to be handled equally.

Even worse: After renaming `.conan/profiles/msvc2019`, the command above worked. So it's not like conan is completely unable to handle relative paths starting with `..`, it just prioritizes questionably and looks in `PROFILE_DIR` first.

After some testing, I also found that `include()` inside of a profile has the same behavior.

---

If feel this is a bad user experience. It makes for very brittle automation scripts (e.g. automatically building system library packages, what I was trying), where they might work in one environment, but fail in another. It also goes against common intuition, expecting all relative paths to work alike. It also goes against common practice of looking in more specific (= more local) places first before looking in more general ones.

---

I'm mostly asking whether this was intended, and hopefully an explanation why it was decided this way. If it isn't intended, I might make this into a bug report, but for now I'm just assuming I'm not getting the full picture.
> Now, with a terminal in ./system, I tried to build conan create llvm -pr ../profiles/msvc2019 -s build_type=Release. That build failed, because it didn't actually take the profile from ./profiles/msvc2019, and instead took a profile from .conan/profiles/msvc2019. (I mostly noticed because some build_requires listed in ./profiles/msvc2019 were missing.)

Indeed I can reproduce your behavior running Conan 1.34.1.

It works for absolute path, but as the documentation says:

> Profiles can be located in different folders. For example, the default <userhome>/.conan/profiles, and be referenced by absolute or relative path:

> $ conan install . --profile /abs/path/to/profile  # abs path
> $ conan install . --profile ./relpath/to/profile  # resolved to current dir
> $ conan install . --profile profile  # resolved to user/.conan/profiles/profile

It doesn't mention ../ but, oblivious, it should work too, because ./msvc2019 works. I would consider it a bug. I'll investigate better. Thanks for reporting your case.



Resolving patch:
diff --git a/conans/client/profile_loader.py b/conans/client/profile_loader.py
--- a/conans/client/profile_loader.py
+++ b/conans/client/profile_loader.py
@@ -82,17 +82,17 @@ def _apply_in_profile_text(self):
 
 
 def get_profile_path(profile_name, default_folder, cwd, exists=True):
-    def valid_path(_profile_path):
+    def valid_path(_profile_path, _profile_name=None):
         if exists and not os.path.isfile(_profile_path):
-            raise ConanException("Profile not found: %s" % _profile_path)
+            raise ConanException("Profile not found: {}".format(_profile_name or _profile_path))
         return _profile_path
 
     if os.path.isabs(profile_name):
         return valid_path(profile_name)
 
-    if profile_name[:2] in ("./", ".\\"):  # local
+    if profile_name[:2] in ("./", ".\\") or profile_name.startswith(".."):  # local
         profile_path = os.path.abspath(os.path.join(cwd, profile_name))
-        return valid_path(profile_path)
+        return valid_path(profile_path, profile_name)
 
     if not os.path.exists(default_folder):
         mkdir(default_folder)


Test patch:
diff --git a/conans/test/functional/configuration/profile_test.py b/conans/test/functional/configuration/profile_test.py
--- a/conans/test/functional/configuration/profile_test.py
+++ b/conans/test/functional/configuration/profile_test.py
@@ -635,3 +635,56 @@ def test_profile_crazy_inheritance(self):
                              compiler.runtime=MD
                              compiler.version=15
                              os=Windows"""), self.client.out)
+
+
+def test_profile_from_cache_path():
+    """ When passing relative folder/profile as profile file, it MUST be used
+        conan install . -pr=profiles/default
+        /tmp/profiles/default MUST be consumed as target profile
+        https://github.com/conan-io/conan/pull/8685
+    """
+    client = TestClient()
+    path = os.path.join(client.cache.profiles_path, "android", "profile1")
+    save(path, "[settings]\nos=Android")
+    client.save({"conanfile.txt": ""})
+    client.run("install . -pr=android/profile1")
+    assert "os=Android" in client.out
+
+
+def test_profile_from_relative_pardir():
+    """ When passing relative ../path as profile file, it MUST be used
+        conan install . -pr=../profiles/default
+        /tmp/profiles/default MUST be consumed as target profile
+    """
+    client = TestClient()
+    client.save({"profiles/default": "[settings]\nos=AIX",
+                 "current/conanfile.txt": ""})
+    with client.chdir("current"):
+        client.run("install . -pr=../profiles/default")
+    assert "os=AIX" in client.out
+
+
+def test_profile_from_relative_dotdir():
+    """ When passing relative ./path as profile file, it MUST be used
+        conan install . -pr=./profiles/default
+        /tmp/profiles/default MUST be consumed as target profile
+    """
+    client = TestClient()
+    client.save({os.path.join("profiles", "default"): "[settings]\nos=AIX",
+                 os.path.join("current", "conanfile.txt"): ""})
+    client.run("install ./current -pr=./profiles/default")
+    assert "os=AIX" in client.out
+
+
+def test_profile_from_temp_absolute_path():
+    """ When passing absolute path as profile file, it MUST be used
+        conan install . -pr=/tmp/profiles/default
+        /tmp/profiles/default MUST be consumed as target profile
+    """
+    client = TestClient()
+    client.save({"profiles/default": "[settings]\nos=AIX",
+                 "current/conanfile.txt": ""})
+    profile_path = os.path.join(client.current_folder, "profiles", "default")
+    recipe_path = os.path.join(client.current_folder, "current", "conanfile.txt")
+    client.run('install "{}" -pr="{}"'.format(recipe_path, profile_path))
+    assert "os=AIX" in client.out
diff --git a/conans/test/unittests/client/profile_loader/profile_loader_test.py b/conans/test/unittests/client/profile_loader/profile_loader_test.py
--- a/conans/test/unittests/client/profile_loader/profile_loader_test.py
+++ b/conans/test/unittests/client/profile_loader/profile_loader_test.py
@@ -355,3 +355,92 @@ def test_include_order(self):
         self.assertEqual(variables, {"MYVAR": "fromProfile2",
                                      "PROFILE_DIR": tmp.replace('\\', '/')})
         self.assertEqual(profile.settings["os"], "fromProfile2")
+
+
+def test_profile_load_absolute_path():
+    """ When passing absolute path as profile file, it MUST be used.
+        read_profile(/abs/path/profile, /abs, /.conan/profiles)
+        /abs/path/profile MUST be consumed as target profile
+    """
+    profile_name = "default"
+    default_profile_folder = os.path.join(temp_folder(), "profiles")
+    default_profile_path = os.path.join(default_profile_folder, profile_name)
+    current_profile_folder = temp_folder()
+    current_profile_path = os.path.join(current_profile_folder, profile_name)
+    default_profile_content = textwrap.dedent("""
+            [env]
+            BORSCHT=BEET SOUP
+        """)
+    current_profile_content = default_profile_content.replace("BEET", "RUSSIAN")
+
+    save(default_profile_path, default_profile_content)
+    save(current_profile_path, current_profile_content)
+
+    profile, variables = read_profile(current_profile_path, current_profile_folder,
+                                      default_profile_folder)
+    assert ({"BORSCHT": "RUSSIAN SOUP"}, {}) == profile.env_values.env_dicts("")
+    assert current_profile_folder.replace("\\", "/") == variables["PROFILE_DIR"]
+
+
+def test_profile_load_relative_path_dot():
+    """ When passing relative ./path as profile file, it MUST be used
+        read_profile(./profiles/profile, /tmp, /.conan/profiles)
+        /tmp/profiles/profile MUST be consumed as target profile
+    """
+    profile_name = "default"
+    default_profile_folder = os.path.join(temp_folder(), "profiles")
+    default_profile_path = os.path.join(default_profile_folder, profile_name)
+    current_profile_folder = temp_folder()
+    current_profile_path = os.path.join(current_profile_folder, profile_name)
+    default_profile_content = textwrap.dedent("""
+            [env]
+            BORSCHT=BEET SOUP
+        """)
+    current_profile_content = default_profile_content.replace("BEET", "RUSSIAN")
+    relative_current_profile_path = "." + os.path.join(os.sep,
+                                                       os.path.basename(current_profile_folder),
+                                                       profile_name)
+
+    save(default_profile_path, default_profile_content)
+    save(current_profile_path, current_profile_content)
+
+    profile, variables = read_profile(relative_current_profile_path,
+                                      os.path.dirname(current_profile_folder),
+                                      default_profile_folder)
+    assert ({"BORSCHT": "RUSSIAN SOUP"}, {}) == profile.env_values.env_dicts("")
+    assert current_profile_folder.replace("\\", "/") == variables["PROFILE_DIR"]
+
+
+def test_profile_load_relative_path_pardir():
+    """ When passing relative ../path as profile file, it MUST be used
+        read_profile(../profiles/profile, /tmp/current, /.conan/profiles)
+        /tmp/profiles/profile MUST be consumed as target profile
+    """
+    profile_name = "default"
+    default_profile_folder = os.path.join(temp_folder(), "profiles")
+    os.mkdir(default_profile_folder)
+    default_profile_path = os.path.join(default_profile_folder, profile_name)
+
+    current_temp_folder = temp_folder()
+    current_profile_folder = os.path.join(current_temp_folder, "profiles")
+    current_running_folder = os.path.join(current_temp_folder, "current")
+
+    os.mkdir(current_profile_folder)
+    os.mkdir(current_running_folder)
+
+    current_profile_path = os.path.join(current_profile_folder, profile_name)
+    default_profile_content = textwrap.dedent("""
+            [env]
+            BORSCHT=BEET SOUP
+        """)
+    current_profile_content = default_profile_content.replace("BEET", "RUSSIAN")
+    relative_current_profile_path = os.pardir + os.path.join(os.sep, "profiles", profile_name)
+
+    save(default_profile_path, default_profile_content)
+    save(current_profile_path, current_profile_content)
+
+    profile, variables = read_profile(relative_current_profile_path,
+                                      current_running_folder,
+                                      default_profile_folder)
+    assert ({"BORSCHT": "RUSSIAN SOUP"}, {}) == profile.env_values.env_dicts("")
+    assert current_profile_folder.replace("\\", "/") == variables["PROFILE_DIR"]