Repository: spack/spack

Issue:
Installation issue: charmpp
Hi,
I am trying to build charmpp.

### Spack version
<!-- Add the output to the command below -->
```console
$ spack --version
0.14.1-365-7b4354b
```

### Steps to reproduce the issue

```console
$ spack install charmpp backend=mpi +cuda arch=linux-centos7-x86_64 %intel ^openmpi
[+] /apps/spack/opt/spack/linux-centos7-x86_64/intel-19.0.5.281/libiconv-1.16-aqfhawrnp7nvv6gyr47sc56yofvpxrj6
[+] /apps/spack/opt/spack/linux-centos7-x86_64/intel-19.0.5.281/pkgconf-1.6.3-joa5olkzyjwezwcmr7lzpmno5xvmvkrr
[+] /apps/spack/opt/spack/linux-centos7-x86_64/intel-19.0.5.281/xz-5.2.5-n2twqlgbv6ojdkarrcri52r44czcyhfe
[+] /apps/spack/opt/spack/linux-centos7-x86_64/intel-19.0.5.281/zlib-1.2.11-kdh3v4z6c36m67oovrg55ba53abj3wev
==> openmpi@4.0.3 : has external module in mpi/openmpi-4.0.3
==> openmpi@4.0.3 : is actually installed in /opt/openmpi-4.0.3
==> openmpi@4.0.3 : already registered in DB
[+] /opt/openmpi-4.0.3
[+] /apps/spack/opt/spack/linux-centos7-x86_64/intel-19.0.5.281/libxml2-2.9.9-ttbguawfhdnehyowlggfumed4iln4ex3
[+] /apps/spack/opt/spack/linux-centos7-x86_64/intel-19.0.5.281/cuda-10.2.89-nfguegtn4hshimv4qrssjco3tidzmmgu
==> 29085: Installing charmpp
==> Using cached archive: /apps/spack/var/spack/cache/_source-cache/archive/ab/ab96198105daabbb8c8bdf370f87b0523521ce502c656cb6cd5b89f69a2c70a8.tar.gz
==> Staging archive: /mnt/resource/root/spack-stage/spack-stage-charmpp-6.10.1-km6ksrbj2gox5cmsykroqtkb2drx6ro5/charm-6.10.1.tar.gz
==> Created stage in /mnt/resource/root/spack-stage/spack-stage-charmpp-6.10.1-km6ksrbj2gox5cmsykroqtkb2drx6ro5
==> No patches needed for charmpp
==> 29085: charmpp: Building charmpp [Package]
==> 29085: charmpp: Executing phase: 'install'
==> Error: OSError: [Errno 2] No such file or directory: '/apps/spack/opt/spack/linux-centos7-x86_64/intel-19.0.5.281/charmpp-6.10.1-km6ksrbj2gox5cmsykroqtkb2drx6ro5/doc'

/apps/spack/var/spack/repos/builtin/packages/charmpp/package.py:249, in install:
        246        try:
        247            os.stat(prefix.doc)
        248        except OSError:
  >>    249            os.remove(prefix.doc)
        250            mkdirp(prefix.doc)

See build log for details:
  /mnt/resource/root/spack-stage/spack-stage-charmpp-6.10.1-km6ksrbj2gox5cmsykroqtkb2drx6ro5/spack-build-out.txt

==> Error: Failed to install charmpp due to ChildError: OSError: [Errno 2] No such file or directory: '/apps/spack/opt/spack/linux-centos7-x86_64/intel-19.0.5.281/charmpp-6.10.1-km6ksrbj2gox5cmsykroqtkb2drx6ro5/doc'
/apps/spack/var/spack/repos/builtin/packages/charmpp/package.py:249, in install:
        246        try:
        247            os.stat(prefix.doc)
        248        except OSError:
  >>    249            os.remove(prefix.doc)
        250            mkdirp(prefix.doc)

See build log for details:
  /mnt/resource/root/spack-stage/spack-stage-charmpp-6.10.1-km6ksrbj2gox5cmsykroqtkb2drx6ro5/spack-build-out.txt
Traceback (most recent call last):
  File "/apps/spack/lib/spack/spack/build_environment.py", line 801, in child_process
    return_value = function()
  File "/apps/spack/lib/spack/spack/installer.py", line 1113, in build_process
    phase(pkg.spec, pkg.prefix)
  File "/apps/spack/lib/spack/spack/package.py", line 108, in phase_wrapper
    phase(spec, prefix)
  File "/apps/spack/var/spack/repos/builtin/packages/charmpp/package.py", line 250, in install
    mkdirp(prefix.doc)
OSError: [Errno 2] No such file or directory: '/apps/spack/opt/spack/linux-centos7-x86_64/intel-19.0.5.281/charmpp-6.10.1-km6ksrbj2gox5cmsykroqtkb2drx6ro5/doc'
```

### Platform and user environment
```commandline
$ uname -a 
Linux ip-AC127404 3.10.0-1062.12.1.el7.x86_64 #1 SMP Tue Feb 4 23:02:59 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux
$ lsb_release -d
CentOS Linux release 7.7.1908 (Core)
``` 

### Additional information

```console
$ spack spec --install-status charmpp backend=mpi +cuda arch=linux-centos7-x86_64 %intel ^openmpi
Input spec
--------------------------------
 -   charmpp%intel backend=mpi +cuda arch=linux-centos7-x86_64
 -       ^openmpi

Concretized
--------------------------------
 -   charmpp@6.10.1%intel@19.0.5.281 backend=mpi build-target=LIBS +cuda~omp~papi+production~pthreads+shared+smp~syncft~tcp~tracing arch=linux-centos7-x86_64
[+]      ^cuda@10.2.89%intel@19.0.5.281 arch=linux-centos7-x86_64
[+]          ^libxml2@2.9.9%intel@19.0.5.281~python arch=linux-centos7-x86_64
[+]              ^libiconv@1.16%intel@19.0.5.281 arch=linux-centos7-x86_64
[+]              ^pkgconf@1.6.3%intel@19.0.5.281 arch=linux-centos7-x86_64
[+]              ^xz@5.2.5%intel@19.0.5.281 arch=linux-centos7-x86_64
[+]              ^zlib@1.2.11%intel@19.0.5.281+optimize+pic+shared arch=linux-centos7-x86_64
[+]      ^openmpi@4.0.3%gcc@9.2.0~cuda+cxx_exceptions fabrics=none ~java~legacylaunchers~memchecker~pmi schedulers=none ~sqlite3~thread_multiple+vt arch=linux-centos7-x86_64
```

Thanks


Discussion:

Resolving patch:
diff --git a/var/spack/repos/builtin/packages/charmpp/package.py b/var/spack/repos/builtin/packages/charmpp/package.py
--- a/var/spack/repos/builtin/packages/charmpp/package.py
+++ b/var/spack/repos/builtin/packages/charmpp/package.py
@@ -58,10 +58,18 @@ class Charmpp(Package):
         "backend",
         default="netlrts",
         values=("mpi", "multicore", "netlrts", "verbs", "gni",
-                "ofi", "pami", "pamilrts"),
+                "ucx", "ofi", "pami", "pamilrts"),
         description="Set the backend to use"
     )
 
+    # Process management interface
+    variant(
+        "pmi",
+        default="none",
+        values=("none", "simplepmi", "slurmpmi", "slurmpmi2", "pmix"),
+        description="The ucx/ofi/gni backends need PMI to run!"
+    )
+
     # Other options
     variant("papi", default=False, description="Enable PAPI integration")
     variant("syncft", default=False, description="Compile with Charm++ fault tolerance support")
@@ -78,29 +86,23 @@ class Charmpp(Package):
     variant("production", default=True, description="Build charm++ with all optimizations")
     variant("tracing", default=False, description="Enable tracing modules")
 
-    # FIXME: backend=mpi also provides mpi, but spack does not support
-    # depends_on("mpi") and provides("mpi") in the same package currently.
-    for b in ['multicore', 'netlrts', 'verbs', 'gni', 'ofi', 'pami',
-              'pamilrts']:
-        provides('mpi@2', when='@6.7.1: build-target=AMPI backend={0}'.format(b))
-        provides('mpi@2', when='@6.7.1: build-target=LIBS backend={0}'.format(b))
-
-    def setup_dependent_build_environment(self, env, dependent_spec):
-        env.set('MPICC',  self.prefix.bin.ampicc)
-        env.set('MPICXX', self.prefix.bin.ampicxx)
-        env.set('MPIF77', self.prefix.bin.ampif77)
-        env.set('MPIF90', self.prefix.bin.ampif90)
-
-    def setup_dependent_package(self, module, dependent_spec):
-        self.spec.mpicc  = self.prefix.bin.ampicc
-        self.spec.mpicxx = self.prefix.bin.ampicxx
-        self.spec.mpifc  = self.prefix.bin.ampif90
-        self.spec.mpif77 = self.prefix.bin.ampif77
-
     depends_on("mpi", when="backend=mpi")
     depends_on("papi", when="+papi")
     depends_on("cuda", when="+cuda")
 
+    depends_on("ucx", when="backend=ucx")
+    depends_on("slurm@:17-11-9-2", when="pmi=slurmpmi")
+    depends_on("slurm@17-11-9-2:", when="pmi=slurmpmi2")
+
+    # FIXME : As of now spack's OpenMPI recipe does not have a PMIx variant
+    # But if users have external installs of OpenMPI with PMIx support, this
+    # will allow them to build charm++ with it.
+    depends_on("openmpi", when="pmi=pmix")
+
+    depends_on("mpi", when="pmi=simplepmi")
+    depends_on("mpi", when="pmi=slurmpmi")
+    depends_on("mpi", when="pmi=slurmpmi2")
+
     # Git versions of Charm++ require automake and autoconf
     depends_on("automake", when="@develop")
     depends_on("autoconf", when="@develop")
@@ -108,11 +110,12 @@ def setup_dependent_package(self, module, dependent_spec):
     conflicts("~tracing", "+papi")
 
     conflicts("backend=multicore", "+smp")
+    conflicts("backend=ucx", when="@:6.9.99")
 
-    def install(self, spec, prefix):
-        target = spec.variants["build-target"].value
-
+    @property
+    def charmarch(self):
         plat = sys.platform
+
         if plat.startswith("linux"):
             plat = "linux"
         elif plat.startswith("win"):
@@ -123,12 +126,13 @@ def install(self, spec, prefix):
             plat = "cnk"
 
         mach = platform.machine()
+
         if mach.startswith("ppc"):
             mach = "ppc"
         elif mach.startswith("arm"):
             mach = "arm"
 
-        comm = spec.variants['backend'].value
+        comm = self.spec.variants['backend'].value
 
         # Define Charm++ version names for various (plat, mach, comm)
         # combinations. Note that not all combinations are supported.
@@ -145,6 +149,7 @@ def install(self, spec, prefix):
             ("linux",   "x86_64",   "netlrts"):     "netlrts-linux-x86_64",
             ("linux",   "x86_64",   "verbs"):       "verbs-linux-x86_64",
             ("linux",   "x86_64",   "ofi"):         "ofi-linux-x86_64",
+            ("linux",   "x86_64",   "ucx"):         "ucx-linux-x86_64",
             ("linux",   "x86_64",   "uth"):         "uth-linux-x86_64",
             ("linux",   "ppc",      "mpi"):         "mpi-linux-ppc",
             ("linux",   "ppc",      "multicore"):   "multicore-linux-ppc",
@@ -167,7 +172,47 @@ def install(self, spec, prefix):
                 "The communication mechanism %s is not supported "
                 "on a %s platform with a %s CPU" %
                 (comm, plat, mach))
-        version = versions[(plat, mach, comm)]
+
+        return versions[(plat, mach, comm)]
+
+    # FIXME: backend=mpi also provides mpi, but spack does not support
+    # depends_on("mpi") and provides("mpi") in the same package currently.
+    # for b in ['multicore', 'netlrts', 'verbs', 'gni', 'ofi', 'pami',
+    #          'pamilrts']:
+    #    provides('mpi@2', when='@6.7.1:
+    #            build-target=AMPI backend={0}'.format(b))
+    #    provides('mpi@2', when='@6.7.1:
+    #            build-target=LIBS backend={0}'.format(b))
+
+    def install(self, spec, prefix):
+
+        if not("backend=mpi" in self.spec) or \
+           not("backend=netlrts" in self.spec):
+            if ("+pthreads" in self.spec):
+                raise InstallError("The pthreads option is only\
+                                    available on the Netlrts and MPI \
+                                    network layers.")
+
+        if ("backend=ucx" in self.spec) or \
+           ("backend=ofi" in self.spec) or \
+           ("backend=gni" in self.spec):
+            if ("pmi=none" in self.spec):
+                raise InstallError("The UCX/OFI/GNI backends need \
+                                    PMI to run. Please add pmi=... \
+                                    Note that PMIx is the preferred \
+                                    option.")
+
+        if ("pmi=simplepmi" in self.spec) or \
+           ("pmi=slurmpmi" in self.spec) or \
+           ("pmi=slurmpmi2" in self.spec):
+            if ("^openmpi" in self.spec):
+                raise InstallError("To use any process management \
+                                    interface other than PMIx, \
+                                    a non OpenMPI based MPI must be \
+                                    present on the system")
+
+        target = spec.variants["build-target"].value
+        builddir = prefix + "/" + str(self.charmarch)
 
         # We assume that Spack's compiler wrappers make this work. If
         # not, then we need to query the compiler vendor from Spack
@@ -176,9 +221,17 @@ def install(self, spec, prefix):
             os.path.basename(self.compiler.cc),
             os.path.basename(self.compiler.fc),
             "-j%d" % make_jobs,
-            "--destination=%s" % prefix,
+            "--destination=%s" % builddir,
         ]
 
+        if "pmi=slurmpmi" in spec:
+            options.append("slurmpmi")
+        if "pmi=slurmpmi2" in spec:
+            options.append("slurmpmi2")
+        if "pmi=pmix" in spec:
+            options.append("ompipmix")
+            options.extend(["--basedir=%s" % spec["openmpi"].prefix])
+
         if 'backend=mpi' in spec:
             # in intelmpi <prefix>/include and <prefix>/lib fails so --basedir
             # cannot be used
@@ -190,6 +243,9 @@ def install(self, spec, prefix):
                 '--libdir={0}'.format(libdir)
                 for libdir in spec["mpi"].libs.directories
             ])
+
+        if "backend=ucx" in spec:
+            options.extend(["--basedir=%s" % spec["ucx"].prefix])
         if "+papi" in spec:
             options.extend(["papi", "--basedir=%s" % spec["papi"].prefix])
         if "+smp" in spec:
@@ -223,11 +279,11 @@ def install(self, spec, prefix):
         # could dissect the build script; the build instructions say
         # this wouldn't be difficult.
         build = Executable(join_path(".", "build"))
-        build(target, version, *options)
+        build(target, self.charmarch, *options)
 
         # Charm++'s install script does not copy files, it only creates
         # symbolic links. Fix this.
-        for dirpath, dirnames, filenames in os.walk(prefix):
+        for dirpath, dirnames, filenames in os.walk(builddir):
             for filename in filenames:
                 filepath = join_path(dirpath, filename)
                 if os.path.islink(filepath):
@@ -239,18 +295,32 @@ def install(self, spec, prefix):
                         os.rename(tmppath, filepath)
                     except (IOError, OSError):
                         pass
-        shutil.rmtree(join_path(prefix, "tmp"))
+        shutil.rmtree(join_path(builddir, "tmp"))
 
-        # A broken 'doc' link in the prefix can break the build.
-        # Remove it and replace it if it is broken.
-        try:
-            os.stat(prefix.doc)
-        except OSError:
-            os.remove(prefix.doc)
-            mkdirp(prefix.doc)
+        if self.spec.satisfies('@6.9.99'):
+            # A broken 'doc' link in the prefix can break the build.
+            # Remove it and replace it if it is broken.
+            try:
+                os.stat(prefix.doc)
+            except OSError:
+                os.remove(prefix.doc)
+                mkdirp(prefix.doc)
 
     @run_after('install')
     @on_package_attributes(run_tests=True)
     def check_build(self):
         make('-C', join_path(self.stage.source_path, 'charm/tests'),
              'test', parallel=False)
+
+    def setup_dependent_build_environment(self, env, dependent_spec):
+        env.set('MPICC',  self.prefix.bin.ampicc)
+        env.set('MPICXX', self.prefix.bin.ampicxx)
+        env.set('MPIF77', self.prefix.bin.ampif77)
+        env.set('MPIF90', self.prefix.bin.ampif90)
+
+    def setup_dependent_package(self, module, dependent_spec):
+        self.spec.mpicc     = self.prefix.bin.ampicc
+        self.spec.mpicxx    = self.prefix.bin.ampicxx
+        self.spec.mpifc     = self.prefix.bin.ampif90
+        self.spec.mpif77    = self.prefix.bin.ampif77
+        self.spec.charmarch = self.charmarch
diff --git a/var/spack/repos/builtin/packages/namd/package.py b/var/spack/repos/builtin/packages/namd/package.py
--- a/var/spack/repos/builtin/packages/namd/package.py
+++ b/var/spack/repos/builtin/packages/namd/package.py
@@ -15,8 +15,11 @@ class Namd(MakefilePackage):
 
     homepage = "http://www.ks.uiuc.edu/Research/namd/"
     url      = "file://{0}/NAMD_2.12_Source.tar.gz".format(os.getcwd())
+    git      = "https://charm.cs.illinois.edu/gerrit/namd.git"
     manual_download = True
 
+    version("develop", branch="master")
+    version('2.14b1', sha256='9407e54f5271b3d3039a5a9d2eae63c7e108ce31b7481e2197c19e1125b43919')
     version('2.13', '9e3323ed856e36e34d5c17a7b0341e38')
     version('2.12', '2a1191909b1ab03bf0205971ad4d8ee9')
 
@@ -26,7 +29,9 @@ class Namd(MakefilePackage):
     variant('interface', default='none', values=('none', 'tcl', 'python'),
             description='Enables TCL and/or python interface')
 
-    depends_on('charmpp')
+    depends_on('charmpp@6.10.1:', when="@2.14b1:")
+    depends_on('charmpp@6.8.2', when="@2.13")
+    depends_on('charmpp@6.7.1', when="@2.12")
 
     depends_on('fftw@:2.99', when="fftw=2")
     depends_on('fftw@3:', when="fftw=3")
@@ -73,17 +78,24 @@ def edit(self, spec, prefix):
             with open('{0}.arch'.format(self.build_directory), 'w') as fh:
                 # this options are take from the default provided
                 # configuration files
-                optims_opts = {
-                    'gcc': '-m64 -O3 -fexpensive-optimizations -ffast-math',
-                    'intel': '-O2 -ip'
-                }
+                # https://github.com/UIUC-PPL/charm/pull/2778
+                if self.spec.satisfies('^charmpp@:6.10.1'):
+                    optims_opts = {
+                        'gcc': '-m64 -O3 -fexpensive-optimizations \
+                                -ffast-math -lpthread',
+                        'intel': '-O2 -ip'}
+                else:
+                    optims_opts = {
+                        'gcc': '-m64 -O3 -fexpensive-optimizations \
+                                -ffast-math',
+                        'intel': '-O2 -ip'}
 
                 optim_opts = optims_opts[self.compiler.name] \
                     if self.compiler.name in optims_opts else ''
 
                 fh.write('\n'.join([
                     'NAMD_ARCH = {0}'.format(self.arch),
-                    'CHARMARCH = ',
+                    'CHARMARCH = {0}'.format(self.spec['charmpp'].charmarch),
                     'CXX = {0.cxx} {0.cxx11_flag}'.format(
                         self.compiler),
                     'CXXOPTS = {0}'.format(optim_opts),


Test patch:
diff --git a/lib/spack/spack/test/cmd/providers.py b/lib/spack/spack/test/cmd/providers.py
--- a/lib/spack/spack/test/cmd/providers.py
+++ b/lib/spack/spack/test/cmd/providers.py
@@ -22,8 +22,7 @@ def test_it_just_runs(pkg):
 
 
 @pytest.mark.parametrize('vpkg,provider_list', [
-    (('mpi',), ['charmpp@6.7.1:',
-                'intel-mpi',
+    (('mpi',), ['intel-mpi',
                 'intel-parallel-studio',
                 'mpich',
                 'mpich@1:',