Repository: spack/spack

Issue:
module generation very slow following ruamel update
### Steps to reproduce

To generate modules for 905 packages takes me just under 4 minutes consistently, until 600955edd4ca22b2d94e725706fc4733d5c4e4d9 was merged, and now it takes ~1 hour! FYI @alalazo 

### Error message

_No response_

### Information on your system

* **Spack:** 0.20.0.dev0 (d265dd248707aff73fae29b67dcc7ad2870a9338)
* **Python:** 3.8.10
* **Platform:** linux-ubuntu20.04-cascadelake
* **Concretizer:** clingo

### General information

- [X] I have run `spack debug report` and reported the version of Spack/Python/Platform
- [X] I have searched the issues of this repo and believe this is not a duplicate
- [X] I have run the failing commands in debug mode and reported the output


Discussion:
I can confirm, it takes ~4 minutes to generate modules for all 900 of my packages if I use the commit right before 600955edd4ca22b2d94e725706fc4733d5c4e4d9, but takes ~60 minutes at 600955edd4ca22b2d94e725706fc4733d5c4e4d9 :(
@alalazo can you get a profile before and after?
I think it's a deepcopy operation on the config file:

https://github.com/spack/spack/blob/9ee2d79de172de14477a78e5d407548a63eea33a/lib/spack/spack/modules/common.py#L174-L175

that became much more honerous with the new version of `ruamel.yaml`. I'll double check and try to confirm, but that seems in line with being worse for "complex" environments - i.e. environments with a large configuration dictionary that is deep copied a lot of times.

Resolving patch:
diff --git a/lib/spack/spack/modules/common.py b/lib/spack/spack/modules/common.py
--- a/lib/spack/spack/modules/common.py
+++ b/lib/spack/spack/modules/common.py
@@ -170,17 +170,12 @@ def merge_config_rules(configuration, spec):
     Returns:
         dict: actions to be taken on the spec passed as an argument
     """
-
-    # Get the top-level configuration for the module type we are using
-    module_specific_configuration = copy.deepcopy(configuration)
-
-    # Construct a dictionary with the actions we need to perform on the spec
-    # passed as a parameter
-
+    # Construct a dictionary with the actions we need to perform on the spec passed as a parameter
+    spec_configuration = {}
     # The keyword 'all' is always evaluated first, all the others are
     # evaluated in order of appearance in the module file
-    spec_configuration = module_specific_configuration.pop("all", {})
-    for constraint, action in module_specific_configuration.items():
+    spec_configuration.update(copy.deepcopy(configuration.get("all", {})))
+    for constraint, action in configuration.items():
         if spec.satisfies(constraint):
             if hasattr(constraint, "override") and constraint.override:
                 spec_configuration = {}
@@ -200,14 +195,14 @@ def merge_config_rules(configuration, spec):
     # configuration
 
     # Hash length in module files
-    hash_length = module_specific_configuration.get("hash_length", 7)
+    hash_length = configuration.get("hash_length", 7)
     spec_configuration["hash_length"] = hash_length
 
-    verbose = module_specific_configuration.get("verbose", False)
+    verbose = configuration.get("verbose", False)
     spec_configuration["verbose"] = verbose
 
     # module defaults per-package
-    defaults = module_specific_configuration.get("defaults", [])
+    defaults = configuration.get("defaults", [])
     spec_configuration["defaults"] = defaults
 
     return spec_configuration


Test patch: