Repository: conan-io/conan

Issue:
[feature] conan config reset
We're using `conan config install <git-repo>` within our CI environment. For our Windows builds we do not use Docker yet, they're just VMs and work like so:
1. Developer pushes changes to a repo
1. CI connects to one of the Windows VMs
1. CI uses `conan config install` to get the latest profiles and configuration for Conan
1. CI builds and tests the repo
1. CI uploads the Conan package
1. CI disconnects. Windows VM waits for the next job.

As you can see Windows jobs are not running within a sandboxed environment. That means when we do `conan config install` and install profileA it will remain there also for future jobs. To work around this we can clear the profiles directory before doing `conan config install`, but this is not as clean as it could be.

Therefore this feature request is to implement something like `conan config reset` or `conan config uninstall`. The command should reset the Conan config to its original state. It would help to have CI mechanisms a bit cleaner and would avoid us to implement any workarounds that may or may not work as intended and with future versions.

- [x] I've read the [CONTRIBUTING guide](https://github.com/conan-io/conan/blob/develop/.github/CONTRIBUTING.md).

[feature] Add option to clean the cache before update via `config_install_interval`
### What is your suggestion?

Hi,

I really like the possibility to update the Conan configuration periodically via the `config_install_interval` variable. However, this doesn't fit our needs to 100%. It would be great if the automatic update functionality could be enhanced to clean the cache (not the package data, but the profiles and hooks) before the configurations are updated. E.g. via a `config_install_clean` variable in the `conan.conf` file. Unfortunately this isn't that easy:
- What shall happen if the configuration is not reachable (e.g. no internet access, folder has been removed, etc.)? Is the cache cleaned but the configuration not applied then?
- Which kind of data shall be cleaned (profiles, hooks, etc.) and which kind of information shall be kept (data folder, ...)?

Another solution would be to provide a hook for that purpose, so a project specific clean up is possible:

```
from conans import tools

def pre_update_config(cache_dir, configs, **kwargs):
    # Project specific clean up of the cache (config provides access to config_install.json structure)
    pass

def post_update_config(cache_dir, valid_configs, invalid_configs, **kwargs):
    # Project specific stuff (valid_configs points to all configs in config_install.json that have been updated properly)
    # Does this make sense if the file has been replaced during the update?
    pass
```

What do you think?

Best regards,
Markus

### Have you read the CONTRIBUTING guide?

- [X] I've read the CONTRIBUTING guide
[feature] Add "force" option to conan remote remove
### What is your suggestion?

Hi,

I just had issue with corrupted remotes.json again. After #16089 I'm getting pretty error message but it still requires manual intervention or additional non-conan actions.

I'm only using these commands for working with remotes:
```
conan remote remove "*"
conan remote add remotename https://.../api/conan/...
conan remote login -p password remotename username
```

As a workaround I've added additional `del /f/s/q "%CONAN_HOME%\remotes.json"` right after `conan remote remove "*"` but I don't really like it. Is it possible to add additional option `--force (-f)` that will completely remove `remotes.json` even if it's unreadable JSON? And maybe create new one with zero remotes.


### Have you read the CONTRIBUTING guide?

- [X] I've read the CONTRIBUTING guide


Discussion:
Hi @monsdar, 
Thank's a lot for the feedback, we'll give a thought at the feature for future developments :)
Hi,
Do you have a plan for this feature? Will be very useful.
@sbudenkov 

I think that is ``conan config init``, that was added some time ago. It seems that this issue was not correctly closed, please check it and let us know.
Hi @memsharded,

Looks like no.
Cmd: _conan config init --force_ doesn't clean files and  folders in .conan directory.
Please check the original post out:
> That means when we do conan config install and install profileA it will remain there also for future jobs. To work around this we can clear the profiles directory before doing conan config install, but this is not as clean as it could be.

Hi @gpxricky 

Thanks very much for your suggestion.
However I don't think that this will be possible. We have actually removed in Conan 2.0 the whole ``config_install_interval`` feature. It was too problematic and complicated.

A different thing might be if we decouple the feature from the automatic config install. Does a ``conan config clean`` command that restores the Conan home to the initial state make sense? Basically a ``conan config clean + conan config install url`` achieves the goal? That might be doable.
Hi @memsharded 

actually I just wanted to open another issue that it is quite impossible now (with enabled `config_install_interval` functionality) to capture STDOUT of Conan as it may be flooded from time to time with messages from the interval update. Therefore a simple `conan remote list` is now complicated to parse :-(.

I can imagine that there is lot of other trouble (failing updates, etc.) that makes it even more complicated. Therefore removing the functionality makes sense and is fine for me. I will try to find another way to realize a Conan 2.0 compatible auto-update. The `conan config clean` may be helpful, but it depends on the meaning of the initial state. E.g. in case it is equal to a non-configured cache (except the storage folder) it is fine for my use case. If it is more or less a `conan config init` then it does not fit my needs.
> actually I just wanted to open another issue that it is quite impossible now (with enabled config_install_interval functionality) to capture STDOUT of Conan as it may be flooded from time to time with messages from the interval update. Therefore a simple conan remote list is now complicated to parse :-(.

This has completely changed already in Conan 2.0. All the output goes to stderr, except the really important bits, that goes to stdout. All commands that expect something valuable will also have a ``--format`` argument to output json or other formats that is easier to parse, it will be explicitly documented in the docs that the terminal output will not be stable and not intended to be parsed, anything intended to be read by machines should be a ``--format`` one.


>  I will try to find another way to realize a Conan 2.0 compatible auto-update.

This sounds to us something more system-level thing, like a ``cron`` job or similar thing. 


> The conan config clean may be helpful, but it depends on the meaning of the initial state. E.g. in case it is equal to a non-configured cache (except the storage folder) it is fine for my use case. If it is more or less a conan config init then it does not fit my needs.

Yes, that would be the idea. To completely remove all config and reset it to the original state after a clean Conan install (except the packages storage).


Hi @memsharded,

actually using a `cron` job or `scheduled task` is the way I will go for.

> Yes, that would be the idea. To completely remove all config and reset it to the original state after a clean Conan install (except the packages storage).

This sounds great and would be a good idea then.
Update: We're looking into adding `conan config clean`, but no promises that it will move forward, it's just being considered with the team now, thanks :)





Patch:
diff --git a/conan/api/conan_api.py b/conan/api/conan_api.py
index 091af879fd7..7b6fd14fec1 100644
--- a/conan/api/conan_api.py
+++ b/conan/api/conan_api.py
@@ -6,7 +6,6 @@
 from conan.api.subapi.local import LocalAPI
 from conan.api.subapi.lockfile import LockfileAPI
 from conan.api.subapi.workspace import WorkspaceAPI
-from conan import conan_version
 from conan.api.subapi.config import ConfigAPI
 from conan.api.subapi.download import DownloadAPI
 from conan.api.subapi.export import ExportAPI
@@ -19,9 +18,9 @@
 from conan.api.subapi.remove import RemoveAPI
 from conan.api.subapi.search import SearchAPI
 from conan.api.subapi.upload import UploadAPI
-from conans.client.migrations import ClientMigrator
 from conan.errors import ConanException
 from conan.internal.paths import get_conan_user_home
+from conans.client.migrations import ClientMigrator
 from conan.internal.model.version_range import validate_conan_version
 
 
@@ -36,12 +35,11 @@ def __init__(self, cache_folder=None):
         self.workspace = WorkspaceAPI(self)
         self.cache_folder = self.workspace.home_folder() or cache_folder or get_conan_user_home()
         self.home_folder = self.cache_folder  # Lets call it home, deprecate "cache"
+        self.migrate()
 
-        # Migration system
-        migrator = ClientMigrator(self.cache_folder, conan_version)
-        migrator.migrate()
-
+        # This API is depended upon by the subsequent ones, it should be initialized first
         self.config = ConfigAPI(self)
+
         self.remotes = RemotesAPI(self)
         self.command = CommandAPI(self)
         # Search recipes by wildcard and packages filtering by configuration
@@ -60,6 +58,24 @@ def __init__(self, cache_folder=None):
         self.lockfile = LockfileAPI(self)
         self.local = LocalAPI(self)
 
-        required_range_new = self.config.global_conf.get("core:required_conan_version")
-        if required_range_new:
-            validate_conan_version(required_range_new)
+        _check_conan_version(self)
+
+    def reinit(self):
+        self.config.reinit()
+        self.remotes.reinit()
+        self.local.reinit()
+
+        _check_conan_version(self)
+
+    def migrate(self):
+        # Migration system
+        # TODO: A prettier refactoring of migrators would be nice
+        from conan import conan_version
+        migrator = ClientMigrator(self.cache_folder, conan_version)
+        migrator.migrate()
+
+
+def _check_conan_version(conan_api):
+    required_range_new = conan_api.config.global_conf.get("core:required_conan_version")
+    if required_range_new:
+        validate_conan_version(required_range_new)
diff --git a/conan/api/subapi/config.py b/conan/api/subapi/config.py
index 02a89445e64..4e56cc4435d 100644
--- a/conan/api/subapi/config.py
+++ b/conan/api/subapi/config.py
@@ -16,11 +16,11 @@
 from conans.client.graph.graph_builder import DepsGraphBuilder
 from conans.client.graph.profile_node_definer import consumer_definer
 from conan.errors import ConanException
-from conan.internal.model.conf import ConfDefinition, BUILT_IN_CONFS
+from conan.internal.model.conf import ConfDefinition, BUILT_IN_CONFS, CORE_CONF_PATTERN
 from conan.internal.model.pkg_type import PackageType
 from conan.internal.model.recipe_ref import RecipeReference
 from conan.internal.model.settings import Settings
-from conans.util.files import load, save
+from conans.util.files import load, save, rmdir, remove
 
 
 class ConfigAPI:
@@ -28,6 +28,7 @@ class ConfigAPI:
     def __init__(self, conan_api):
         self.conan_api = conan_api
         self._new_config = None
+        self._cli_core_confs = None
 
     def home(self):
         return self.conan_api.cache_folder
@@ -40,6 +41,7 @@ def install(self, path_or_url, verify_ssl, config_type=None, args=None,
         requester = self.conan_api.remotes.requester
         configuration_install(cache_folder, requester, path_or_url, verify_ssl, config_type=config_type, args=args,
                               source_folder=source_folder, target_folder=target_folder)
+        self.conan_api.reinit()
 
     def install_pkg(self, ref, lockfile=None, force=False, remotes=None, profile=None):
         ConanOutput().warning("The 'conan config install-pkg' is experimental",
@@ -100,6 +102,7 @@ def install_pkg(self, ref, lockfile=None, force=False, remotes=None, profile=Non
         config_versions = {ref.split("/", 1)[0]: ref for ref in config_versions}
         config_versions[pkg.pref.ref.name] = pkg.pref.repr_notime()
         save(config_version_file, json.dumps({"config_version": list(config_versions.values())}))
+        self.conan_api.reinit()
         return pkg.pref
 
     def get(self, name, default=None, check_type=None):
@@ -114,11 +117,19 @@ def global_conf(self):
         configuration defined with the new syntax as in profiles, this config will be composed
         to the profile ones and passed to the conanfiles.conf, which can be passed to collaborators
         """
+        # Lazy loading
         if self._new_config is None:
-            cache_folder = self.conan_api.cache_folder
-            self._new_config = self.load_config(cache_folder)
+            self._new_config = ConfDefinition()
+            self._populate_global_conf()
         return self._new_config
 
+    def _populate_global_conf(self):
+        cache_folder = self.conan_api.cache_folder
+        new_config = self.load_config(cache_folder)
+        self._new_config.update_conf_definition(new_config)
+        if self._cli_core_confs is not None:
+            self._new_config.update_conf_definition(self._cli_core_confs)
+
     @staticmethod
     def load_config(home_folder):
         # Do not document yet, keep it private
@@ -191,3 +202,35 @@ def appending_recursive_dict_update(d, u):
             appending_recursive_dict_update(settings, settings_user)
 
         return Settings(settings)
+
+    def clean(self):
+        contents = os.listdir(self.home())
+        packages_folder = self.global_conf.get("core.cache:storage_path") or os.path.join(self.home(), "p")
+        for content in contents:
+            content_path = os.path.join(self.home(), content)
+            if content_path == packages_folder or content == "version.txt":
+                continue
+            ConanOutput().debug(f"Removing {content_path}")
+            if os.path.isdir(content_path):
+                rmdir(content_path)
+            else:
+                remove(content_path)
+        self.conan_api.reinit()
+        # CHECK: This also generates a remotes.json that is not there after a conan profile show?
+        self.conan_api.migrate()
+
+    def set_core_confs(self, core_confs):
+        confs = ConfDefinition()
+        for c in core_confs:
+            if not CORE_CONF_PATTERN.match(c):
+                raise ConanException(f"Only core. values are allowed in --core-conf. Got {c}")
+        confs.loads("\n".join(core_confs))
+        confs.validate()
+        self._cli_core_confs = confs
+        # Last but not least, apply the new configuration
+        self.conan_api.reinit()
+
+    def reinit(self):
+        if self._new_config is not None:
+            self._new_config.clear()
+            self._populate_global_conf()
diff --git a/conan/api/subapi/local.py b/conan/api/subapi/local.py
index bf078441eeb..e745152ab4c 100644
--- a/conan/api/subapi/local.py
+++ b/conan/api/subapi/local.py
@@ -118,3 +118,6 @@ def inspect(self, conanfile_path, remotes, lockfile, name=None, version=None, us
         conanfile = app.loader.load_named(conanfile_path, name=name, version=version, user=user,
                                           channel=channel, remotes=remotes, graph_lock=lockfile)
         return conanfile
+
+    def reinit(self):
+        self.editable_packages = EditablePackages(self._conan_api.home_folder)
diff --git a/conan/api/subapi/remotes.py b/conan/api/subapi/remotes.py
index 45a08d1342a..33e85253bb9 100644
--- a/conan/api/subapi/remotes.py
+++ b/conan/api/subapi/remotes.py
@@ -35,6 +35,9 @@ def __init__(self, conan_api):
         # Wraps an http_requester to inject proxies, certs, etc
         self._requester = ConanRequester(self.conan_api.config.global_conf, self.conan_api.cache_folder)
 
+    def reinit(self):
+        self._requester = ConanRequester(self.conan_api.config.global_conf, self.conan_api.cache_folder)
+
     def list(self, pattern=None, only_enabled=True):
         """
         Obtain a list of ``Remote`` objects matching the pattern.
diff --git a/conan/cli/command.py b/conan/cli/command.py
index 5447203b37b..4a7e80971b5 100644
--- a/conan/cli/command.py
+++ b/conan/cli/command.py
@@ -4,7 +4,6 @@
 
 from conan.api.output import ConanOutput
 from conan.errors import ConanException
-from conan.internal.model.conf import CORE_CONF_PATTERN
 
 
 class OnceArgument(argparse.Action):
@@ -126,14 +125,7 @@ def parse_args(self, args=None, namespace=None):
             ConanOutput().error("The --lockfile-packages arg is private and shouldn't be used")
         global_conf = self._conan_api.config.global_conf
         if args.core_conf:
-            from conan.internal.model.conf import ConfDefinition
-            confs = ConfDefinition()
-            for c in args.core_conf:
-                if not CORE_CONF_PATTERN.match(c):
-                    raise ConanException(f"Only core. values are allowed in --core-conf. Got {c}")
-            confs.loads("\n".join(args.core_conf))
-            confs.validate()
-            global_conf.update_conf_definition(confs)
+            self._conan_api.config.set_core_confs(args.core_conf)
 
         # TODO: This might be even better moved to the ConanAPI so users without doing custom
         #  commands can benefit from it
diff --git a/conan/cli/commands/config.py b/conan/cli/commands/config.py
index 9aeaa2e5f69..8454d1401b3 100644
--- a/conan/cli/commands/config.py
+++ b/conan/cli/commands/config.py
@@ -1,3 +1,4 @@
+from conan.api.input import UserInput
 from conan.api.model import Remote
 from conan.api.output import cli_out_write
 from conan.cli.command import conan_command, conan_subcommand, OnceArgument
@@ -132,3 +133,12 @@ def config_show(conan_api, parser, subparser, *args):
     args = parser.parse_args(*args)
 
     return conan_api.config.show(args.pattern)
+
+
+@conan_subcommand()
+def config_clean(conan_api, parser, subparser, *args):
+    """
+    Clean the configuration files in the Conan home folder. (Keeping installed packages)
+    """
+    parser.parse_args(*args)
+    conan_api.config.clean()
diff --git a/conan/internal/model/conf.py b/conan/internal/model/conf.py
index 065af309854..4f035f04afd 100644
--- a/conan/internal/model/conf.py
+++ b/conan/internal/model/conf.py
@@ -705,3 +705,6 @@ def loads(self, text, profile=False):
     def validate(self):
         for conf in self._pattern_confs.values():
             conf.validate()
+
+    def clear(self):
+        self._pattern_confs.clear()
diff --git a/conans/client/migrations.py b/conans/client/migrations.py
index 82997ba4e9e..026ef8b819b 100644
--- a/conans/client/migrations.py
+++ b/conans/client/migrations.py
@@ -3,7 +3,6 @@
 import textwrap
 
 from conan.api.output import ConanOutput
-from conan.api.subapi.config import ConfigAPI
 from conan.internal.default_settings import migrate_settings_file
 from conans.migrations import Migrator
 from conans.util.dates import timestamp_now
@@ -76,6 +75,7 @@ def migrate(home_folder):
 
 
 def _migrate_pkg_db_lru(cache_folder, old_version):
+    from conan.api.subapi.config import ConfigAPI
     config = ConfigAPI.load_config(cache_folder)
     storage = config.get("core.cache:storage_path") or os.path.join(cache_folder, "p")
     db_filename = os.path.join(storage, 'cache.sqlite3')


Test patch:
diff --git a/test/integration/command/config_test.py b/test/integration/command/config_test.py
index 17c552ec4bc..c84ea3e3680 100644
--- a/test/integration/command/config_test.py
+++ b/test/integration/command/config_test.py
@@ -2,7 +2,10 @@
 import os
 import textwrap
 
+import pytest
+
 from conan.api.conan_api import ConanAPI
+from conan.test.assets.genconanfile import GenConanfile
 from conan.internal.model.conf import BUILT_IN_CONFS
 from conan.test.utils.test_files import temp_folder
 from conan.test.utils.tools import TestClient
@@ -201,3 +204,67 @@ def test_config_show():
     tc.run("config show zlib/*:foo")
     assert "zlib/*:user.mycategory:foo" in tc.out
     assert "zlib/*:user.myothercategory:foo" in tc.out
+
+
+@pytest.mark.parametrize("storage_path", [None, "p", "../foo"])
+def test_config_clean(storage_path):
+    tc = TestClient(light=True)
+    absolut_storage_path = os.path.abspath(os.path.join(tc.current_folder, storage_path)) if storage_path else os.path.join(tc.cache_folder, "p")
+
+    storage = f"core.cache:storage_path={storage_path}" if storage_path else ""
+    tc.save_home({"global.conf": f"core.upload:retry=7\n{storage}",
+                  "extensions/compatibility/mycomp.py": "",
+                  "extensions/commands/cmd_foo.py": "",
+                  })
+
+    tc.run("profile detect --name=foo")
+    tc.run("remote add bar http://fakeurl")
+
+    tc.save({"conanfile.py": GenConanfile("pkg", "0.1")})
+    tc.run("create .")
+
+    assert os.path.exists(absolut_storage_path)
+
+    tc.run("config clean")
+    tc.run("profile list")
+    assert "foo" not in tc.out
+    tc.run("remote list")
+    assert "bar" not in tc.out
+    tc.run("config show core.upload:retry")
+    assert "7" not in tc.out
+    assert not os.path.exists(os.path.join(tc.cache_folder, "extensions"))
+    assert os.path.exists(absolut_storage_path)
+
+
+def test_config_reinit():
+    custom_global_conf = "core.upload:retry=7"
+    global_conf_folder = temp_folder()
+    with open(os.path.join(global_conf_folder, "global.conf"), "w") as f:
+        f.write(custom_global_conf)
+
+    cache_folder = temp_folder()
+    conan_api = ConanAPI(cache_folder=cache_folder)
+    # Ensure reinitialization does not invalidate references
+    config_api = conan_api.config
+    assert config_api.global_conf.get("core.upload:retry", check_type=int) != 7
+
+    conan_api.config.install(global_conf_folder, verify_ssl=False)
+    # Already has an effect, the config installation reinitializes the config
+    assert config_api.global_conf.get("core.upload:retry", check_type=int) == 7
+
+
+def test_config_reinit_core_conf():
+    tc = TestClient(light=True)
+    tc.save_home({"extensions/commands/cmd_foo.py": textwrap.dedent("""
+        import json
+        from conan.cli.command import conan_command
+        from conan.api.output import ConanOutput
+
+        @conan_command()
+        def foo(conan_api, parser, *args, **kwargs):
+            ''' Foo '''
+            parser.parse_args(*args)
+            ConanOutput().info(f"Retry: {conan_api.config.global_conf.get('core.upload:retry', check_type=int)}")
+    """)})
+    tc.run("foo -cc core.upload:retry=7")
+    assert "Retry: 7" in tc.out