Repository: spack/spack

Issue:
Spack detects duplicate compiler modules on Piz Daint
When running `spack compiler find` on Piz Daint, Spack detects duplicate entries for `cce`. 

### Steps to reproduce the issue

This issue is reproducible on Piz Daint (CSCS) by:
```console
$ git clone https://github.com/spack/spack.git
$ cd spack
$ . share/spack/setup-env.sh
$ spack compiler list
==> Available compilers
-- cce cnl7-any -------------------------------------------------
cce@9.0.2  cce@9.0.2  cce@9.0.1  cce@9.0.1  cce@8.7.9

-- gcc cnl7-any -------------------------------------------------
gcc@8.3.0  gcc@8.2.0  gcc@8.1.0  gcc@7.3.0  gcc@6.1.0

-- gcc sles15-x86_64 --------------------------------------------
gcc@7.4.1

-- intel cnl7-any -----------------------------------------------
intel@19.0.1.144  intel@18.0.2.199  intel@18.0.1.163  intel@17.0.4.196

-- pgi cnl7-any -------------------------------------------------
pgi@19.7.0  pgi@19.5.0  pgi@19.4.0
```
As can bee seen in the first line there are duplicate entries for ` cce@9.0.2` and `cce@9.0.1`.

### Error Message

There is no error message, only duplicate entries that stem from an incorrect detection of the versions that are available via module files:
```console
$ module av cce

------------------------------------------------------------ /opt/cray/pe/modulefiles ------------------------------------------------------------
cce/8.7.9          cce/9.0.1          cce/9.0.1-classic  cce/9.0.2(default) cce/9.0.2-classic

```

### Information on your system

* **Spack:** 0.14.2-1444-4b48f7039
* **Python:** 3.6.8
* **Platform:** cray-sles15-haswell

### Additional information
Running compiler detection in debug mode we get:
```console
spack -d compiler find
==> [2020-06-16-16:43:35.739429, 5313] '/bin/bash' '--norc' '--noprofile' '-lc' 'echo $CRAY_CPU_TARGET'
==> [2020-06-16-16:43:35.746306, 5313] default to front-end architecture
==> [2020-06-16-16:43:35.792600, 5313] Imported compiler from built-in commands
==> [2020-06-16-16:43:35.794860, 5313] Imported compiler from built-in commands
==> [2020-06-16-16:43:36.376073, 5313] '/usr/bin/gfortran-7' '-dumpversion'
==> [2020-06-16-16:43:36.377433, 5313] '/usr/bin/gfortran' '-dumpversion'
==> [2020-06-16-16:43:36.380151, 5313] '/usr/bin/gfortran' '-dumpversion'
==> [2020-06-16-16:43:36.378778, 5313] '/usr/bin/gfortran-7' '-dumpversion'
==> [2020-06-16-16:43:36.381488, 5313] '/usr/bin/g++-7' '--version'
==> [2020-06-16-16:43:36.387805, 5313] '/usr/bin/gcc' '--version'
==> [2020-06-16-16:43:36.392334, 5313] '/usr/bin/gfortran' '--version'
==> [2020-06-16-16:43:36.395216, 5313] '/usr/bin/gfortran' '--version'
==> [2020-06-16-16:43:36.386379, 5313] '/usr/bin/gcc-7' '--version'
==> [2020-06-16-16:43:36.404775, 5313] '/usr/bin/gfortran-7' '-dumpfullversion'
==> [2020-06-16-16:43:36.382759, 5313] '/usr/bin/g++' '--version'
==> [2020-06-16-16:43:36.389104, 5313] '/usr/bin/gfortran-7' '--version'
==> [2020-06-16-16:43:36.429171, 5313] '/usr/bin/g++-7' '-dumpversion'
==> [2020-06-16-16:43:36.435213, 5313] '/usr/bin/gfortran' '-dumpfullversion'
==> [2020-06-16-16:43:36.393870, 5313] '/usr/bin/gfortran-7' '--version'
==> [2020-06-16-16:43:36.449949, 5313] '/usr/bin/gfortran-7' '-dumpfullversion'
==> [2020-06-16-16:43:36.419942, 5313] '/usr/bin/gfortran' '-dumpfullversion'
==> [2020-06-16-16:43:36.430545, 5313] '/usr/bin/gcc' '-dumpversion'
==> [2020-06-16-16:43:36.453562, 5313] '/usr/bin/g++' '-dumpversion'
==> [2020-06-16-16:43:36.451955, 5313] '/usr/bin/gcc-7' '-dumpversion'
==> [2020-06-16-16:43:36.468703, 5313] '/usr/bin/g++-7' '-dumpfullversion'
==> [2020-06-16-16:43:36.484837, 5313] '/usr/bin/g++' '-dumpfullversion'
==> [2020-06-16-16:43:36.492467, 5313] '/usr/bin/gcc-7' '-dumpfullversion'
==> [2020-06-16-16:43:36.494332, 5313] '/usr/bin/gcc' '-dumpfullversion'
==> [2020-06-16-16:43:36.501806, 5313] Couldn't get version for compiler /usr/bin/gfortran-7
==> [2020-06-16-16:43:36.501930, 5313] Couldn't get version for compiler /usr/bin/gfortran
==> [2020-06-16-16:43:36.501981, 5313] Couldn't get version for compiler /usr/bin/gfortran-7
==> [2020-06-16-16:43:36.502022, 5313] Couldn't get version for compiler /usr/bin/gfortran
==> [2020-06-16-16:43:36.502059, 5313] Error while executing candidate compiler /usr/bin/cc
TypeError: expected string or bytes-like object
==> [2020-06-16-16:43:36.535807, 5313] Added 18 new compilers to /users/culpo/.spack/cray/compilers.yaml
    pgi@19.7.0  pgi@19.4.0        intel@18.0.2.199  intel@17.0.4.196  gcc@8.2.0  gcc@7.4.1  gcc@6.1.0  cce@9.0.2  cce@9.0.1
    pgi@19.5.0  intel@19.0.1.144  intel@18.0.1.163  gcc@8.3.0         gcc@8.1.0  gcc@7.3.0  cce@9.0.2  cce@9.0.1  cce@8.7.9
==> [2020-06-16-16:43:36.536854, 5313] Compilers are defined in the following files:
    /users/culpo/.spack/cray/compilers.yaml
```

- [x] I have run `spack debug report` and reported the version of Spack/Python/Platform
- [x] I have searched the issues of this repo and believe this is not a duplicate
- [x] I have run the failing commands in debug mode and reported the output

<!-- We encourage you to try, as much as possible, to reduce your problem to the minimal example that still reproduces the issue. That would help us a lot in fixing it quickly and effectively!

If you want to ask a question about the tool (how to use it, what it can currently do, etc.), try the `#general` channel on our Slack first. We have a welcoming community and chances are you'll get your reply faster and without opening an issue.

Other than that, thanks for taking the time to contribute to Spack! -->



Discussion:
#16593 was the most recent change to how we handle Cray, maybe you could try reverting that and see if it works for you? @becker33 may want to take a look at this.
Thanks for the pointer @adamjstewart I think if we're lucky it might be simply some regex in the Cray Backend platform that needs update. I'll look into that today.
@alalazo it looks like the issue is that there are two versions of each of those compilers from different modules (for the "regular" and "classic" versions of the compiler). The "classic" version of the compiler uses the old Cray frontend and compiler options, the "regular" one uses a new clang-based frontend and clang-compatible compiler options.
@becker33 You're definitely right, even though I'm still puzzled by the decision to have the same version on the compilers :man_shrugging: I think we can solve this either by tweaking:

https://github.com/spack/spack/blob/ad18933ef400ecbb879feebb32368ab7cc898493/lib/spack/spack/operating_systems/cray_backend.py#L145

or by deduplicating the entries (in case we just want to keep the new Clang based compiler).
I think we

1. tweak the regex to detect the classic as `9.0.0-classic`
2. tweak the version checks in `lib/spack/spack/compilers/cce.py` to only use the new flags for versions >= 9.0 if "classic" isn't in the version.

Resolving patch:
diff --git a/lib/spack/spack/compilers/cce.py b/lib/spack/spack/compilers/cce.py
--- a/lib/spack/spack/compilers/cce.py
+++ b/lib/spack/spack/compilers/cce.py
@@ -32,9 +32,14 @@ class Cce(Compiler):
                   'f77': 'cce/ftn',
                   'fc': 'cce/ftn'}
 
+    @property
+    def is_clang_based(self):
+        version = self.version
+        return version >= ver('9.0') and 'classic' not in str(version)
+
     @property
     def version_argument(self):
-        if self.version >= ver('9.0'):
+        if self.is_clang_based:
             return '--version'
         return '-V'
 
@@ -50,19 +55,19 @@ def debug_flags(self):
 
     @property
     def openmp_flag(self):
-        if self.version >= ver('9.0'):
+        if self.is_clang_based:
             return '-fopenmp'
         return "-h omp"
 
     @property
     def cxx11_flag(self):
-        if self.version >= ver('9.0'):
+        if self.is_clang_based:
             return '-std=c++11'
         return "-h std=c++11"
 
     @property
     def c99_flag(self):
-        if self.version >= ver('9.0'):
+        if self.is_clang_based:
             return '-std=c99'
         elif self.version >= ver('8.4'):
             return '-h std=c99,noconform,gnu'
@@ -75,7 +80,7 @@ def c99_flag(self):
 
     @property
     def c11_flag(self):
-        if self.version >= ver('9.0'):
+        if self.is_clang_based:
             return '-std=c11'
         elif self.version >= ver('8.5'):
             return '-h std=c11,noconform,gnu'
diff --git a/lib/spack/spack/operating_systems/cray_backend.py b/lib/spack/spack/operating_systems/cray_backend.py
--- a/lib/spack/spack/operating_systems/cray_backend.py
+++ b/lib/spack/spack/operating_systems/cray_backend.py
@@ -142,7 +142,9 @@ def detect_version(self, detect_version_args):
         compiler_name = detect_version_args.id.compiler_name
         compiler_cls = spack.compilers.class_for_compiler_name(compiler_name)
         output = modulecmd('avail', compiler_cls.PrgEnv_compiler)
-        version_regex = r'(%s)/([\d\.]+[\d])' % compiler_cls.PrgEnv_compiler
+        version_regex = r'({0})/([\d\.]+[\d]-?[\w]*)'.format(
+            compiler_cls.PrgEnv_compiler
+        )
         matches = re.findall(version_regex, output)
         version = tuple(version for _, version in matches)
         compiler_id = detect_version_args.id


Test patch:
diff --git a/lib/spack/spack/test/compilers/basics.py b/lib/spack/spack/test/compilers/basics.py
--- a/lib/spack/spack/test/compilers/basics.py
+++ b/lib/spack/spack/test/compilers/basics.py
@@ -355,12 +355,20 @@ def test_arm_flags():
 
 
 def test_cce_flags():
+    supported_flag_test("version_argument", "--version", "cce@9.0.1")
+    supported_flag_test("version_argument", "-V", "cce@9.0.1-classic")
+    supported_flag_test("openmp_flag", "-fopenmp", "cce@9.0.1")
+    supported_flag_test("openmp_flag", "-h omp", "cce@9.0.1-classic")
     supported_flag_test("openmp_flag", "-h omp", "cce@1.0")
+    supported_flag_test("cxx11_flag", "-std=c++11", "cce@9.0.1")
+    supported_flag_test("cxx11_flag", "-h std=c++11", "cce@9.0.1-classic")
     supported_flag_test("cxx11_flag", "-h std=c++11", "cce@1.0")
     unsupported_flag_test("c99_flag", "cce@8.0")
+    supported_flag_test("c99_flag", "-std=c99", "cce@9.0.1")
     supported_flag_test("c99_flag", "-h c99,noconform,gnu", "cce@8.1")
     supported_flag_test("c99_flag", "-h std=c99,noconform,gnu", "cce@8.4")
     unsupported_flag_test("c11_flag", "cce@8.4")
+    supported_flag_test("c11_flag", "-std=c11", "cce@9.0.1")
     supported_flag_test("c11_flag", "-h std=c11,noconform,gnu", "cce@8.5")
     supported_flag_test("cc_pic_flag",  "-h PIC", "cce@1.0")
     supported_flag_test("cxx_pic_flag", "-h PIC", "cce@1.0")