Repository: spack/spack

Issue:
Remove SPACK_DISABLE_LOCAL_CONFIG
1. It's only confusing since we allow `$user_config_path` as a variable, which is used in `bootstrap.yaml`, meaning that, although config from `~/.spack` is not read, it still is potentially referred and written to in places.
2. With `SPACK_DISABLE_LOCAL_CONFIG`, the first writable config scope for spack is `$spack/etc/spack`, it's only a matter of time until people start opening issues about that.

Another one-liner solution is to use `$user_cache_path` in bootstrap.yaml's defaults, but this goes counter to the original goal of being able to reuse the same bootstrap store across multiple spack versions (or rather: package repos) using a different misc cache folder. So we should probably not pursue that.

After this PR, ci scripts would have to use:

```
SPACK_SYSTEM_CONFIG_PATH="$(mktemp -d)"
SPACK_USER_CONFIG_PATH="$workspace/.spack"
SPACK_USER_CACHE_PATH="$workspace/.spack"
```

Before this PR, they would have to do:

```
SPACK_DISABLE_LOCAL_CONFIG=yes_please
SPACK_USER_CONFIG_PATH="$workspace/.spack"
SPACK_USER_CACHE_PATH="$workspace/.spack"
```

so it's not an inconvenience.

In the end it's important we don't solve this problem through even more ad-hoc solutions, but rather we should reduce the complexity introduced in #26735, of which I wasn't a fan from the start... Any new feature that's introduced is something we'd have to live with in 0.17, and that was exactly the reason my original version of #26735 was *very* minimal.


Discussion:
The time to talk about this was #26735, which we merged yesterday.

Will accept bug fixes for SPACK_DISABLE_LOCAL_CONFIG if it’s not working properly.
Really?!

I've issued concerns in that PR

> I think you're forgetting about the bootstrap:root:~/.spack/bootstrap default. That makes me wonder how useful SPACK_DISABLE_LOCAL_CONFIG is, since the user would still have to set bootstrap:root to some persistent location to avoid bootstrapping every invocation

Both @alalazo and @tjfulle agreed to that

> I have the same concern as @haampie but other than that LGTM

> I concur with @haampie RE SPACK_DISABLE_LOCAL_CONFIG.

So it seems this is entirely your decision, and you've not listened.

The thing I pointed out there was almost identical to the issue described in this PR, it's just that nobody had thought of it, because it was very much rushed.
Also @mathstuf raised a concern

> Will that work for the case where I want spack compiler find to be able to write its discoveries somewhere?

which is legitimate, and is item 2 mentioned in this PR.
FWIW, I don't like the `SPACK_DISABLE_LOCAL_CONFIG` variable.  It's not really true to the name as it disables the user config and system config.  My two cents:

- `SPACK_USER_CONFIG_PATH` is necessary.  The user must be able to control where spack reads/writes user data.  The cache and bootstrap dirs should continue to live here.  Ie, if a user sets this, Spack should *never* write to `~/.spack`.
- `SPACK_SYSTEM_CONFIG_PATH` is wholly unnecessary and, to me, just muddies the waters.  It should be replaced by `SPACK_DISABLE_SYSTEM_CONFIG_SCOPE` which is self explanatory.  If a system provides an alternative config scope, use the `-C` flag.

With the above, compilers, cache, and bootstrap are written to the directory of their choice.  If they want to have a cache and bootstrap shared between different `SPACK_USER_CONFIG_PATH`s they can create appropriate symlinks.


Ok reopening for discussion.  I want there to be one single way for the user to disable configuration from the host machine.  That's the point of `SPACK_DISABLE_LOCAL_CONFIG`.  I don't think we should have to do this:

```
SPACK_USER_CONFIG_PATH=$workspace/.spack
SPACK_SYSTEM_CONFIG_PATH=$workspace/.spack
```

Because you have to:

1. Come up with a location. You don't actually know a good one on a random cloud instance, and nobody seemed to like the tmpdir idea.
2. Set a variable per location to disable.

Disabling the config scopes altogether is much simpler -- there's no location to set; the user doesn't have to think about it or come up with anything.

`user_config_path` does not occur anywhere in the default configs.  So if you've added it in, asked to disable configs and referred to it still, that doesn't make sense. My suggestion would be to remove the `user_config_path` variable altogether and tell people to use `user_cache_path` if they want this behavior.
> It's not really true to the name as it disables the user config and system config.

Both of these are local to the host machine.
FWIW, I have made progress without the `SPACK_DISABLE_LOCAL_CONFIG` setting by using the suggestion in the description here. It wasn't sufficient (as-is) and if I have to set the user and system paths anyways…not sure what it is gaining us.
@tgamblin - thanks for the clarification on the meaning of the local config variable.  That makes more sense.  But, can spack *ever* realistically disable a local config?  I don't think so.  By local config I mean a local directory containing config files, cache, and bootstrapping data, etc.  As far as I can tell, Spack needs to be able to read/write these.   I see nothing wrong with dropping all that in `~/.spack` since that is the defacto standard for Linux.  Why muddy it by having an additional cache setting variable?  I would just expect the location of the user config path to be customizable, which is reasonable since Spack's target audience are Linux and HPC users where the home directory is often mounted on different machines.

The system config path *should not* be configurable.  It should be in `/etc/spack`, period.  I should, however, be able to disable it.  I'd actually prefer to have to opt in, but that's another discussion.
> I see nothing wrong with dropping all that in ~/.spack since that is the defacto standard for Linux.

CI machines on platforms without containers need to ensure that global state like this is *not* touched in a writable manner, so disabling it is important (I've added symlinks from `~/.spack -> /dev/null` to ensure that anything trying ends up falling on its face).

> It should be in /etc/spack, period.

Distros and package managers would love to make this `/usr/local/etc` (e.g., ports) or `/opt/brew/etc` (HomeBrew) if they end up putting Spack into `/usr/local`.

Though the utility of ports or HomeBrew shipping Spack is admittedly weird :) .
@mathstuf - I agree.  So spack should honor `export SPACK_USER_CONFIG_PATH=/dev/null`.  Done.  I'm not sure it currently does, it may still try to bootstrap, for example.

you're right, The system path may change depending on the conventions for the system.  My point is users should not tell spack where the system path is, that's weird.  They should be able to opt out of it.
Well, the thing is I still need user configuration (the compiler detection needs written somewhere after all), so just being able to redirect it to be under the CI root is sufficient (as long as everything actually behaves).
@tjfulle:

> I see nothing wrong with dropping all that in ~/.spack since that is the defacto standard for Linux.

FWIW, there is no de facto standard for Linux -- or at least there is not just one.  `~/.spack` is probably the most used convention, but XDG would have us stick this in `~/.config/spack`, and 50% of people seem to have no idea that that is a thing.  So I would say there are at least two standards for config.

> Why muddy it by having an additional cache setting variable?

If we are talking about standards, XDG actually *does* break out a separate `~/.cache` directory for exactly the reason we did in #26735 -- cache is different from config and it's for different types of data.

You can see the issue in your comment here:

>  I agree. So spack should honor export SPACK_USER_CONFIG_PATH=/dev/null. Done

If you do this, you're not actually done, because if `~/.spack` is the default location for caches *and* config, and this variable controls both, then setting it to `/dev/null` gives us no default place to write persistent caches.  This is why I broke out the cache directory and made them settable separately.

Basically: 
* Spack *always* needs to have a writable directory for things like repo indices, bootstrap data, test output, etc.
* It does *not* actually always need a user configuration directory - if it lacks one it will try to write to `$spack/etc/spack`, unless there is an active environment, in which case it'll write to `spack.yaml`

So it makes sense to me to allow the user to do one very simple thing to guarantee an isolated env -- set `SPACK_DISABLE_LOCAL_CONFIG`, and not actually worry about the caches.  Working on making that last part true -- it seems like there are lingering ~/.spack references, as mentioned by @mathstuf and @trws in #26735, and we do still need to fix the misc_cache to handle multiple Spack instances.

Once all that is done you should not actually NEED to set anything but `SPACK_DISABLE_LOCAL_CONFIG` to get rid of these directories -- unless you really need to be specific.

Resolving patch:
diff --git a/lib/spack/spack/monitor.py b/lib/spack/spack/monitor.py
--- a/lib/spack/spack/monitor.py
+++ b/lib/spack/spack/monitor.py
@@ -28,6 +28,7 @@
 import spack.config
 import spack.hash_types as ht
 import spack.main
+import spack.paths
 import spack.store
 import spack.util.path
 import spack.util.spack_json as sjson
@@ -143,7 +144,8 @@ def setup_save(self):
             return
 
         save_dir = spack.util.path.canonicalize_path(
-            spack.config.get('config:monitor_dir', '~/.spack/reports/monitor'))
+            spack.config.get('config:monitor_dir', spack.paths.default_monitor_path)
+        )
 
         # Name based on timestamp
         now = datetime.now().strftime('%Y-%m-%d-%H-%M-%S-%s')
diff --git a/lib/spack/spack/paths.py b/lib/spack/spack/paths.py
--- a/lib/spack/spack/paths.py
+++ b/lib/spack/spack/paths.py
@@ -89,8 +89,11 @@ def _get_user_cache_path():
 #: junit, cdash, etc. reports about builds
 reports_path = os.path.join(user_cache_path, "reports")
 
+#: installation test (spack test) output
+default_test_path = os.path.join(user_cache_path, "test")
+
 #: spack monitor analysis directories
-monitor_path = os.path.join(reports_path, "monitor")
+default_monitor_path = os.path.join(reports_path, "monitor")
 
 #: git repositories fetched to compare commits to versions
 user_repos_cache_path = os.path.join(user_cache_path, 'git_repos')
diff --git a/lib/spack/spack/util/path.py b/lib/spack/spack/util/path.py
--- a/lib/spack/spack/util/path.py
+++ b/lib/spack/spack/util/path.py
@@ -30,7 +30,6 @@
     'spack': spack.paths.prefix,
     'user': getpass.getuser(),
     'tempdir': tempfile.gettempdir(),
-    'user_config_path': spack.paths.user_config_path,
     'user_cache_path': spack.paths.user_cache_path,
 }
 
@@ -80,7 +79,6 @@ def substitute_config_variables(path):
     - $spack             The Spack instance's prefix
     - $tempdir           Default temporary directory returned by tempfile.gettempdir()
     - $user              The current user's username
-    - $user_config_path  The user configuration directory (~/.spack, unless overridden)
     - $user_cache_path   The user cache directory (~/.spack, unless overridden)
 
     These are substituted case-insensitively into the path, and users can


Test patch:
diff --git a/lib/spack/spack/install_test.py b/lib/spack/spack/install_test.py
--- a/lib/spack/spack/install_test.py
+++ b/lib/spack/spack/install_test.py
@@ -13,6 +13,7 @@
 import llnl.util.tty as tty
 
 import spack.error
+import spack.paths
 import spack.util.prefix
 import spack.util.spack_json as sjson
 from spack.spec import Spec
@@ -41,7 +42,8 @@ def get_escaped_text_output(filename):
 
 def get_test_stage_dir():
     return spack.util.path.canonicalize_path(
-        spack.config.get('config:test_stage', '~/.spack/test'))
+        spack.config.get('config:test_stage', spack.paths.default_test_path)
+    )
 
 
 def get_all_test_suites():
diff --git a/lib/spack/spack/test/config.py b/lib/spack/spack/test/config.py
--- a/lib/spack/spack/test/config.py
+++ b/lib/spack/spack/test/config.py
@@ -432,13 +432,6 @@ def test_substitute_user(mock_low_high_config):
     )
 
 
-def test_substitute_user_config(mock_low_high_config):
-    user_config_path = spack.paths.user_config_path
-    assert user_config_path + '/baz' == spack_path.canonicalize_path(
-        '$user_cache_path/baz'
-    )
-
-
 def test_substitute_user_cache(mock_low_high_config):
     user_cache_path = spack.paths.user_cache_path
     assert user_cache_path + '/baz' == spack_path.canonicalize_path(