Repository: spack/spack

Issue:
Version 46ab939 is unable to create an environment with spack.yaml file 
<!-- Explain, in a clear and concise way, the command you ran and the result you were trying to achieve.
Example: "I ran `spack find` to list all the installed packages and ..." -->

When I run a newer version of spack to recreate an environment that used to work I get the following error:

spack -d env create foo ~/spack.git/var/spack/environments/lbann-dev-power9le/spack.yaml
==> Error: 'syaml_str' object has no attribute 'lc'


### Steps to reproduce the issue

```console
$ spack -d env create foo ~/spack.git/var/spack/environments/lbann-dev-power9le/spack.yaml
...
```

### Error Message

<!-- If Spack reported an error, provide the error message. If it did not report an error but the output appears incorrect, provide the incorrect output. If there was no error message and no output but the result is incorrect, describe how it does not match what you expect. -->
```console
==> Error: 'syaml_str' object has no attribute 'lc'
```

### Information on your system

<!-- Please include the output of `spack debug report` -->

This failed on Lassen.

vanessen@lassen709 [spack_test.git] %> spack debug report
* **Spack:** 0.14.2-1172-46ab939
* **Python:** 3.7.2
* **Platform:** linux-rhel7-power9le

<!-- If you have any relevant configuration detail (custom `packages.yaml` or `modules.yaml`, etc.) you can add that here as well. -->

### Additional information

<!-- These boxes can be checked by replacing [ ] with [x] or by clicking them after submitting the issue. -->
- [x] I have run `spack debug report` and reported the version of Spack/Python/Platform
- [x] I have searched the issues of this repo and believe this is not a duplicate
- [x] I have run the failing commands in debug mode and reported the output

<!-- We encourage you to try, as much as possible, to reduce your problem to the minimal example that still reproduces the issue. That would help us a lot in fixing it quickly and effectively!

If you want to ask a question about the tool (how to use it, what it can currently do, etc.), try the `#general` channel on our Slack first. We have a welcoming community and chances are you'll get your reply faster and without opening an issue.

Other than that, thanks for taking the time to contribute to Spack! -->



Discussion:
@scheibelp Here is an issue

Here is the spack.yaml file.

```spack:
  concretization: together
  specs:
  
# These packages should go away when spack fixes its environments
  - aluminum
  - hydrogen
  - dihydrogen
  - cereal
  - clara
  - cnpy
  - conduit

  - hwloc
  - opencv
  - zlib
  - cudnn
  - cub
  - cuda
  - nccl
  - py-numpy
  - py-protobuf
  - py-setuptools

  
# These are required
  - catch2
  - cmake
  - ninja
  - python
  - py-pytest
  - gcc
  packages:
    all:
      providers:
        mpi: [spectrum-mpi@rolling-release arch=linux-rhel7-power9le]
        lapack: [openblas threads=openmp]
        blas: [openblas threasd=openmp]
      buildable: true
      version: []
      paths: {}
      modules: {}
      compiler: [gcc@7.3.0 arch=linux-rhel7-broadwell, gcc@7.3.0 arch=linux-rhel7-haswell,
        gcc@7.3.1 arch=linux-rhel7-power9le, gcc@7.3.1 arch=linux-rhel7-power8le]
      target: []
    'cmake:':
      buildable: true
      variants: ~openssl ~ncurses
      version: [3.14.5]
      paths:
        cmake@3.14.5 arch=linux-rhel7-power9le: /usr/tce/packages/cmake/cmake-3.14.5
      target: []
      compiler: []
      modules: {}
      providers: {}
    'cuda:':
      buildable: false
      version: [10.1.243]
      modules:
        cuda@10.1.243 arch=linux-rhel7-power9le: cuda/10.1.243
      target: []
      compiler: []
      providers: {}
      paths: {}
    'cudnn:':
      buildable: true
      version: [7.6.5.32-10.1-linux-ppc64le]
      target: []
      compiler: []
      modules: {}
      providers: {}
      paths: {}
    'gcc:':
      buildable: false
      version: [7.3.1]
      modules:
        gcc@7.3.1 arch=linux-rhel7-power9le: gcc/7.3.1
      target: []
      compiler: []
      providers: {}
      paths: {}
    'hwloc:':
      buildable: false
      version: [2.0.2]
      paths:
        hwloc@2.0.2 arch=linux-rhel7-power9le: /usr/lib64/libhwloc.so
      target: []
      compiler: []
      modules: {}
      providers: {}
  'openblas:':
      buildable: true
      variants: threads=openmp ~avx2 ~avx512
      version: [0.3.6]
      target: []
      compiler: []
      modules: {}
      providers: {}
      paths: {}
    'opencv:':
      buildable: true
      variants: build_type=RelWithDebInfo ~calib3d+core~cuda~dnn~eigen+fast-math~features2d~flann~gtk+highgui+imgproc~ipp~ipp_iw~jasper~java+jpeg~lapack~ml~opencl~opencl_svm~openclamdblas~openclamdfft~openmp+png+powerpc~pthreads_pf~python~qt+shared~stitching~superres+tiff~ts~video~videoio~videostab+vsx~vtk+zlib
      version: [4.1.0]
      target: []
      compiler: []
      modules: {}
      providers: {}
      paths: {}
    'python:':
      buildable: true
      variants: +shared ~readline ~zlib ~bz2 ~lzma ~pyexpat
      version: [3.7.2]
      modules:
        python@3.7.2 arch=linux-rhel7-power9le: python/3.7.2
      target: []
      compiler: []
      providers: {}
      paths: {}
    'rdma-core:':
      buildable: false
      version: [20]
      paths:
        rdma-core@20 arch=linux-rhel7-power9le: /usr
      target: []
      compiler: []
      modules: {}
      providers: {}
    'spectrum-mpi:':
      buildable: false
      version: [rolling-release]
      paths:
        spectrum-mpi@rolling-release %gcc@7.3.1 arch=linux-rhel7-power9le: /usr/tce/packages/spectrum-mpi/spectrum-mpi-rolling-release-gcc-7.3.1
      target: []
      compiler: []
      modules: {}
      providers: {}
    'cereal:':
      buildable: true
      version: [1.2.2]
      target: []
      compiler: []
      modules: {}
      providers: {}
      paths: {}
    'conduit:':
      buildable: true
      variants: ~doc~doxygen+hdf5~hdf5_compat+mpi+python+shared~silo
      version: [0.5.1]
      target: []
      compiler: []
      modules: {}
      providers: {}
      paths: {}
    'cnpy:':
      buildable: true
      variants: build_type=RelWithDebInfo
      version: [master]
      target: []
      compiler: []
      modules: {}
      providers: {}
      paths: {}
    'cub:':
      buildable: true
      version: [1.8.0]
      target: []
      compiler: []
      modules: {}
      providers: {}
      paths: {}
   'nccl:':
      buildable: true
      version: [2.5.7-1]
      target: []
      compiler: []
      modules: {}
      providers: {}
      paths: {}
    'protobuf:':
      buildable: true
      variants: build_type=Release +shared
      version: [3.10.0]
      target: []
      compiler: []
      modules: {}
      providers: {}
      paths: {}
    'py-numpy:':
      buildable: true
      version: [1.16.2]
      target: []
      compiler: []
      modules: {}
      providers: {}
      paths: {}
    'py-protobuf:':
      buildable: true
      variants: +cpp
      version: [3.10.0]
      target: []
      compiler: []
      modules: {}
      providers: {}
      paths: {}
    'zlib:':
      buildable: true
      version: [1.2.11]
      target: []
      compiler: []
      modules: {}
      providers: {}
      paths: {}
    aluminum:
      buildable: true
      version: [0.3.3]
      variants: +gpu+nccl+mpi_cuda
      providers: {}
      paths: {}
      modules: {}
      compiler: []
      target: []
    hydrogen:
      buildable: true
      version: [develop]
      variants: +shared +int64 +al ~half +openmp_blas +cuda
      providers: {}
      paths: {}
      modules: {}
      compiler: []
      target: []
    dihydrogen:
      buildable: true
      version: [master]
      variants: +shared +al +openmp ~half +cuda +legacy
      providers: {}
      paths: {}
      modules: {}
      compiler: []
      target: []
  compilers:
  - compiler:
      environment: {unset: []}
      extra_rpaths: []
      flags: {}
      modules: []
      operating_system: rhel7
      paths:
        cc: /usr/tce/packages/gcc/gcc-7.3.1/bin/gcc
        cxx: /usr/tce/packages/gcc/gcc-7.3.1/bin/g++
        f77: /usr/tce/packages/gcc/gcc-7.3.1/bin/gfortran
        fc: /usr/tce/packages/gcc/gcc-7.3.1/bin/gfortran
      spec: gcc@7.3.1
      target: ppc64le
  - compiler:
      environment: {}
      extra_rpaths: []
      flags: {}
      modules: []
      operating_system: rhel7
      paths:
        cc: /usr/tce/packages/gcc/gcc-7.3.0/bin/gcc
        cxx: /usr/tce/packages/gcc/gcc-7.3.0/bin/g++
        f77: /usr/tce/packages/gcc/gcc-7.3.0/bin/gfortran
        fc: /usr/tce/packages/gcc/gcc-7.3.0/bin/gfortran
      spec: gcc@7.3.0
      target: x86_64
  modules:
    'enable:':
    - tcl
    - lmod
    'lmod:':
      hash_length: 3
      core_compilers:
      - gcc@7.3.0
      - gcc@7.3.1
      naming_scheme: ${PACKAGE}/${VERSION}-${COMPILERNAME}-${COMPILERVER}
      blacklist:
      - '%gcc@4.8'
      - '%gcc@4.9.3'
      hierarchy:
      - mpi
      - lapack
      all:
        autoload: direct
        suffixes:
          ^openblas: openblas
          ^netlib-lapack: netlib
        filter:
          environment_blacklist: [CPATH, LIBRARY_PATH]
        conflict: []
        load: []
        environment:
          unset: []
      ^python:
        autoload: direct
        filter:
          environment_blacklist: []
        conflict: []
        load: []
        environment:
          unset: []
      verbose: false
      whitelist: []
      blacklist_implicits: false
    tcl:
      hash_length: 3
      core_compilers:
      - gcc@7.3.0
      - gcc@7.3.1
      naming_scheme: ${PACKAGE}/${VERSION}-${COMPILERNAME}-${COMPILERVER}
      whitelist:
      - gcc
      blacklist:
      - '%gcc@4.8'
      - '%gcc@4.9.3'
      all:
        autoload: direct
        suffixes:
          ^openblas: openblas
          ^netlib-lapack: netlib
        filter:
          environment_blacklist: [CPATH, LIBRARY_PATH]
        conflict: []
        load: []
        environment:
          unset: []
      ^python:
        autoload: direct
        filter:
          environment_blacklist: []
        conflict: []
        load: []
        environment:
          unset: []
      verbose: false
      blacklist_implicits: false
  view: true
```
The property "naming_scheme" doesn't exist - replace:

```
naming_scheme: ${PACKAGE}/${VERSION}-${COMPILERNAME}-${COMPILERVER}
```

with

```
projections:
  all: ${PACKAGE}/${VERSION}-${COMPILERNAME}-${COMPILERVER}
```

The error this generates does not have a line number for some reason, so the code generating the error message for this issue was itself failing - https://github.com/spack/spack/pull/16765 fixes that.
Okay, that seemed to fix it.  I am curious, I am sure that I got the field naming_scheme from the spack documentation at some earlier time.  Is it correct that it used to be valid?

Brian C. Van Essen
vanessen1@llnl.gov
(w) 925-422-9300
(c) 925-290-5470





> On May 22, 2020, at 10:51 AM, Peter Scheibel <notifications@github.com> wrote:
> 
> 
> The property "naming_scheme" doesn't exist - replace:
> 
> naming_scheme: ${PACKAGE}/${VERSION}-${COMPILERNAME}-${COMPILERVER}
> 
> with
> 
> projections:
>   all: ${PACKAGE}/${VERSION}-${COMPILERNAME}-${COMPILERVER}
> 
> The error this generates does not have a line number for some reason, so the code generating the error message for this issue was itself failing - #16765 fixes that.
> 
> —
> You are receiving this because you authored the thread.
> Reply to this email directly, view it on GitHub, or unsubscribe.
> 


It looks like https://github.com/spack/spack/pull/16629 did in fact remove `naming_scheme` and replace it with `projections`.

@becker33 will this cause issues for all existing environments or am I missing something (e.g. is there an automatic conversion mechanism that might not be getting applied here)?
Hitting the same thing:

```
[quellyn@darwin-fe3 1.1]$ spack concretize -f
==> Error: /projects/shavano/dev/quellyn/dw/DWX86P/spack-environments/DWX86P_19.7/1.1/ENV/common/modules.yaml:11: '{name}/{version}-{compiler.name}-{compiler.version}' is not of type 'object'
```

[quellyn@darwin-fe3 1.1]$ spack debug report
* **Spack:** 0.14.2-1194-7449bf8
* **Python:** 3.6.8
* **Platform:** linux-centos7-haswell

@scheibelp @becker33 Can a mechanism like the one in #16526 be helpful here to update the configuration files? In case I can refactor that PR to have a schema update separate from GCC detection.
@scheibelp yes that was not backwards compatible. I'll create a PR to allow backwards compatibility.
Based on https://github.com/spack/spack/issues/16754#issuecomment-634121326, I'll reassign this issue to @becker33. #16765 addresses the error message issue but I think this issue should stay open if we plan to fix it by allowing backwards compatibility.

Resolving patch:
diff --git a/lib/spack/spack/modules/common.py b/lib/spack/spack/modules/common.py
--- a/lib/spack/spack/modules/common.py
+++ b/lib/spack/spack/modules/common.py
@@ -397,8 +397,13 @@ def __init__(self, spec):
     @property
     def projections(self):
         """Projection from specs to module names"""
-        projections = self.module.configuration().get(
-            'projections', self.default_projections)
+        # backwards compatiblity for naming_scheme key
+        conf = self.module.configuration()
+        if 'naming_scheme' in conf:
+            default = {'all': conf['naming_scheme']}
+        else:
+            default = self.default_projections
+        projections = conf.get('projections', default)
 
         # Ensure the named tokens we are expanding are allowed, see
         # issue #2884 for reference
diff --git a/lib/spack/spack/schema/modules.py b/lib/spack/spack/schema/modules.py
--- a/lib/spack/spack/schema/modules.py
+++ b/lib/spack/spack/schema/modules.py
@@ -17,7 +17,8 @@
 #: THIS NEEDS TO BE UPDATED FOR EVERY NEW KEYWORD THAT
 #: IS ADDED IMMEDIATELY BELOW THE MODULE TYPE ATTRIBUTE
 spec_regex = r'(?!hierarchy|core_specs|verbose|hash_length|whitelist|' \
-             r'blacklist|projections|core_compilers|all)(^\w[\w-]*)'
+             r'blacklist|projections|naming_scheme|core_compilers|all)' \
+             r'(^\w[\w-]*)'
 
 #: Matches an anonymous spec, i.e. a spec without a root name
 anonymous_spec_regex = r'^[\^@%+~]'
@@ -94,6 +95,9 @@
                 'type': 'boolean',
                 'default': False
             },
+            'naming_scheme': {
+                'type': 'string'  # Can we be more specific here?
+            },
             'projections': projections_scheme,
             'all': module_file_configuration,
         }


Test patch:
diff --git a/lib/spack/spack/test/data/modules/tcl/naming_scheme.yaml b/lib/spack/spack/test/data/modules/tcl/naming_scheme.yaml
new file mode 100644
--- /dev/null
+++ b/lib/spack/spack/test/data/modules/tcl/naming_scheme.yaml
@@ -0,0 +1,4 @@
+enable:
+  - tcl
+tcl:
+  naming_scheme: '{name}/{version}-{compiler.name}'
diff --git a/lib/spack/spack/test/modules/tcl.py b/lib/spack/spack/test/modules/tcl.py
--- a/lib/spack/spack/test/modules/tcl.py
+++ b/lib/spack/spack/test/modules/tcl.py
@@ -142,6 +142,20 @@ def test_blacklist(self, modulefile_content, module_configuration):
         assert len([x for x in content if 'is-loaded' in x]) == 1
         assert len([x for x in content if 'module load ' in x]) == 1
 
+    def test_naming_scheme_compat(self, factory, module_configuration):
+        """Tests backwards compatibility for naming_scheme key"""
+        module_configuration('naming_scheme')
+
+        # Test we read the expected configuration for the naming scheme
+        writer, _ = factory('mpileaks')
+        expected = {
+            'all': '{name}/{version}-{compiler.name}'
+        }
+
+        assert writer.conf.projections == expected
+        projection = writer.spec.format(writer.conf.projections['all'])
+        assert projection in writer.layout.use_name
+
     def test_projections_specific(self, factory, module_configuration):
         """Tests reading the correct naming scheme."""