Repository: spack/spack

Issue:
User settings for install_tree:padding propagate to bootstrapping when they shouldn't
### Steps to reproduce

Originally reported by @elliottslaughter on Slack

When adding a large value for padding, for instance:
```console
$ spack config add config:install_tree:padded_length:512
```
the bootstrapping logic goes into an infinite recursion trying to install `patchelf`.

### Error message

```console
 spack solve zlib
==> Fetching https://mirror.spack.io/bootstrap/github-actions/v0.4/build_cache/linux-centos7-x86_64-gcc-10.2.1-patchelf-0.16.1-p72zyan5wrzuabtmzq7isa5mzyh6ahdp.spec.json
==> Fetching https://mirror.spack.io/bootstrap/github-actions/v0.4/build_cache/linux-centos7-x86_64/gcc-10.2.1/patchelf-0.16.1/linux-centos7-x86_64-gcc-10.2.1-patchelf-0.16.1-p72zyan5wrzuabtmzq7isa5mzyh6ahdp.spack
==> Installing "patchelf@=0.16.1%gcc@=10.2.1 ldflags="-static-libstdc++ -static-libgcc"  build_system=autotools arch=linux-centos7-x86_64" from a buildcache
==> Fetching https://mirror.spack.io/bootstrap/github-actions/v0.4/build_cache/linux-centos7-x86_64-gcc-10.2.1-patchelf-0.16.1-p72zyan5wrzuabtmzq7isa5mzyh6ahdp.spec.json
==> Fetching https://mirror.spack.io/bootstrap/github-actions/v0.4/build_cache/linux-centos7-x86_64/gcc-10.2.1/patchelf-0.16.1/linux-centos7-x86_64-gcc-10.2.1-patchelf-0.16.1-p72zyan5wrzuabtmzq7isa5mzyh6ahdp.spack
==> Installing "patchelf@=0.16.1%gcc@=10.2.1 ldflags="-static-libstdc++ -static-libgcc"  build_system=autotools arch=linux-centos7-x86_64" from a buildcache
==> Fetching https://mirror.spack.io/bootstrap/github-actions/v0.4/build_cache/linux-centos7-x86_64-gcc-10.2.1-patchelf-0.16.1-p72zyan5wrzuabtmzq7isa5mzyh6ahdp.spec.json
==> Fetching https://mirror.spack.io/bootstrap/github-actions/v0.4/build_cache/linux-centos7-x86_64/gcc-10.2.1/patchelf-0.16.1/linux-centos7-x86_64-gcc-10.2.1-patchelf-0.16.1-p72zyan5wrzuabtmzq7isa5mzyh6ahdp.spack
==> Installing "patchelf@=0.16.1%gcc@=10.2.1 ldflags="-static-libstdc++ -static-libgcc"  build_system=autotools arch=linux-centos7-x86_64" from a buildcache
==> Fetching https://mirror.spack.io/bootstrap/github-actions/v0.4/build_cache/linux-centos7-x86_64-gcc-10.2.1-patchelf-0.16.1-p72zyan5wrzuabtmzq7isa5mzyh6ahdp.spec.json
==> Fetching https://mirror.spack.io/bootstrap/github-actions/v0.4/build_cache/linux-centos7-x86_64/gcc-10.2.1/patchelf-0.16.1/linux-centos7-x86_64-gcc-10.2.1-patchelf-0.16.1-p72zyan5wrzuabtmzq7isa5mzyh6ahdp.spack
==> Installing "patchelf@=0.16.1%gcc@=10.2.1 ldflags="-static-libstdc++ -static-libgcc"  build_system=autotools arch=linux-centos7-x86_64" from a buildcache
==> Fetching https://mirror.spack.io/bootstrap/github-actions/v0.4/build_cache/linux-centos7-x86_64-gcc-10.2.1-patchelf-0.16.1-p72zyan5wrzuabtmzq7isa5mzyh6ahdp.spec.json
==> Fetching https://mirror.spack.io/bootstrap/github-actions/v0.4/build_cache/linux-centos7-x86_64/gcc-10.2.1/patchelf-0.16.1/linux-centos7-x86_64-gcc-10.2.1-patchelf-0.16.1-p72zyan5wrzuabtmzq7isa5mzyh6ahdp.spack
==> Installing "patchelf@=0.16.1%gcc@=10.2.1 ldflags="-static-libstdc++ -static-libgcc"  build_system=autotools arch=linux-centos7-x86_64" from a buildcache
==> Fetching https://mirror.spack.io/bootstrap/github-actions/v0.4/build_cache/linux-centos7-x86_64-gcc-10.2.1-patchelf-0.16.1-p72zyan5wrzuabtmzq7isa5mzyh6ahdp.spec.json
==> Fetching https://mirror.spack.io/bootstrap/github-actions/v0.4/build_cache/linux-centos7-x86_64/gcc-10.2.1/patchelf-0.16.1/linux-centos7-x86_64-gcc-10.2.1-patchelf-0.16.1-p72zyan5wrzuabtmzq7isa5mzyh6ahdp.spack
==> Installing "patchelf@=0.16.1%gcc@=10.2.1 ldflags="-static-libstdc++ -static-libgcc"  build_system=autotools arch=linux-centos7-x86_64" from a buildcache
...
```

The error occurs because of several factors:
1. We add RPATHS that are not really needed to binaries. This causes e.g. `patchelf` to be "relocated" and not just copied into place
2. We have a fast relocation procedure that doesn't use `patchelf`, if the new path is shorter than the original one. When it isn't... it falls back to use patchelf.
3. The user configuration for p[adding propagates to bootstrapping, but that isn't strictly needed (users care for padding when they install software for their own use, not for Spack's internal use)


### Information on your system

* **Spack:** 0.21.0.dev0 (82aa27f5a5bb000301efda633a0704544d1eadc0)
* **Python:** 3.8.10
* **Platform:** linux-ubuntu20.04-icelake
* **Concretizer:** clingo


### General information

- [X] I have run `spack debug report` and reported the version of Spack/Python/Platform
- [X] I have searched the issues of this repo and believe this is not a duplicate
- [X] I have run the failing commands in debug mode and reported the output


Discussion:
@haampie fyi

Resolving patch:
diff --git a/lib/spack/spack/bootstrap/config.py b/lib/spack/spack/bootstrap/config.py
--- a/lib/spack/spack/bootstrap/config.py
+++ b/lib/spack/spack/bootstrap/config.py
@@ -155,13 +155,13 @@ def _ensure_bootstrap_configuration() -> Generator:
     with spack.environment.no_active_environment():
         with spack.platforms.prevent_cray_detection(), spack.platforms.use_platform(
             spack.platforms.real_host()
-        ), spack.repo.use_repositories(spack.paths.packages_path), spack.store.use_store(
-            bootstrap_store_path
-        ):
+        ), spack.repo.use_repositories(spack.paths.packages_path):
             # Default configuration scopes excluding command line
             # and builtin but accounting for platform specific scopes
             config_scopes = _bootstrap_config_scopes()
-            with spack.config.use_configuration(*config_scopes):
+            with spack.config.use_configuration(*config_scopes), spack.store.use_store(
+                bootstrap_store_path, extra_data={"padded_length": 0}
+            ):
                 # We may need to compile code from sources, so ensure we
                 # have compilers for the current platform
                 _add_compilers_if_missing()


Test patch:
diff --git a/lib/spack/spack/test/bootstrap.py b/lib/spack/spack/test/bootstrap.py
--- a/lib/spack/spack/test/bootstrap.py
+++ b/lib/spack/spack/test/bootstrap.py
@@ -23,20 +23,29 @@ def active_mock_environment(mutable_config, mutable_mock_env_path):
 
 @pytest.mark.regression("22294")
 def test_store_is_restored_correctly_after_bootstrap(mutable_config, tmpdir):
-    # Prepare a custom store path. This should be in a writeable location
-    # since Spack needs to initialize the DB.
+    """Tests that the store is correctly swapped during bootstrapping, and restored afterward."""
     user_path = str(tmpdir.join("store"))
-    # Reassign global variables in spack.store to the value
-    # they would have at Spack startup.
-    spack.store.reinitialize()
-    # Set the custom user path
-    spack.config.set("config:install_tree:root", user_path)
-
-    # Test that within the context manager we use the bootstrap store
-    # and that outside we restore the correct location
-    with spack.bootstrap.ensure_bootstrap_configuration():
-        assert spack.store.root == spack.bootstrap.config.store_path()
-    assert spack.store.root == user_path
+    with spack.store.use_store(user_path):
+        assert spack.store.root == user_path
+        assert spack.config.config.get("config:install_tree:root") == user_path
+        with spack.bootstrap.ensure_bootstrap_configuration():
+            assert spack.store.root == spack.bootstrap.config.store_path()
+        assert spack.store.root == user_path
+        assert spack.config.config.get("config:install_tree:root") == user_path
+
+
+@pytest.mark.regression("38963")
+def test_store_padding_length_is_zero_during_bootstrapping(mutable_config, tmpdir):
+    """Tests that, even though padded length is set in user config, the bootstrap store maintains
+    a padded length of zero.
+    """
+    user_path = str(tmpdir.join("store"))
+    with spack.store.use_store(user_path, extra_data={"padded_length": 512}):
+        assert spack.config.config.get("config:install_tree:padded_length") == 512
+        with spack.bootstrap.ensure_bootstrap_configuration():
+            assert spack.store.root == spack.bootstrap.config.store_path()
+            assert spack.config.config.get("config:install_tree:padded_length") == 0
+        assert spack.config.config.get("config:install_tree:padded_length") == 512
 
 
 @pytest.mark.parametrize(