Repository: spack/spack

Issue:
hpx: Invalid default variant
The max_cpu_count variant for hpx is set to 64 by default.  If that value doesn't match the actual number of cpus this can result in runtime failures like:

```
mpirun -np 1 ./build/test_future --hpx:use-process-mask
hpx::init: hpx::exception caught: Currently, HPX_HAVE_MAX_CPU_COUNT is set to 64 while your system has 96 processing units. Please reconfigure HPX with -DHPX_WITH_MAX_CPU_COUNT=96 (or higher) to increase the maximal CPU count supported by HPX.: HPX(invalid_status)
```

Does the correct cpu count need to be determined at install time (making binary relocation ungeneralizable for hpx), or is there a flag that can be added at build or runtime to force hpx to allow a sub-optimal cpu count? Alternatively, is there a way to determine the correct value from inside the spack package and use that for configuration?

@albestro  @hkaiser  @msimberg  @teonnik


Discussion:
HPX itself actually already defaults to not setting a max CPU count and uses dynamic bitsets instead for CPU masks (that's what the max count is primarily for). The default in the spack package doesn't reflect the HPX default. From previous testing performance was not affected by using dynamic bitsets so changing the default in the spack package would probably make sense. @hkaiser do any of the new schedulers care more about using static or dynamic bitsets?

https://github.com/spack/spack/blob/e9149cfc3cfc94bd66aa0e0901ce562cf8b75713/var/spack/repos/builtin/packages/hpx/package.py#L61-L66 would need to be changed to allow some special value to represent no limit. In HPX's CMake logic it's represented by an empty string, but maybe something like `none` is a better way to represent it in the spack package?
I think currently HPX still uses a maximum of 64 cores as the default internal value (if nothing else was specified). However @msimberg is right that we could (and should) remove this restriction. I will prepare a PR to be integrated with the upcoming HPX V1.10.0 release.

Resolving patch:
diff --git a/var/spack/repos/builtin/packages/hpx/package.py b/var/spack/repos/builtin/packages/hpx/package.py
--- a/var/spack/repos/builtin/packages/hpx/package.py
+++ b/var/spack/repos/builtin/packages/hpx/package.py
@@ -60,9 +60,9 @@ class Hpx(CMakePackage, CudaPackage, ROCmPackage):
 
     variant(
         "max_cpu_count",
-        default="64",
+        default="auto",
         description="Max number of OS-threads for HPX applications",
-        values=lambda x: isinstance(x, str) and x.isdigit(),
+        values=lambda x: isinstance(x, str) and (x.isdigit() or x == "auto"),
     )
 
     instrumentation_values = ("apex", "google_perftools", "papi", "valgrind")
@@ -224,6 +224,9 @@ def instrumentation_args(self):
     def cmake_args(self):
         spec, args = self.spec, []
 
+        format_max_cpu_count = lambda max_cpu_count: (
+            "" if max_cpu_count == "auto" else max_cpu_count
+        )
         args += [
             self.define("HPX_WITH_CXX{0}".format(spec.variants["cxxstd"].value), True),
             self.define_from_variant("HPX_WITH_MALLOC", "malloc"),
@@ -237,7 +240,10 @@ def cmake_args(self):
             self.define("HPX_WITH_NETWORKING", "networking=none" not in spec),
             self.define("HPX_WITH_PARCELPORT_TCP", "networking=tcp" in spec),
             self.define("HPX_WITH_PARCELPORT_MPI", "networking=mpi" in spec),
-            self.define_from_variant("HPX_WITH_MAX_CPU_COUNT", "max_cpu_count"),
+            self.define(
+                "HPX_WITH_MAX_CPU_COUNT",
+                format_max_cpu_count(spec.variants["max_cpu_count"].value),
+            ),
             self.define_from_variant("HPX_WITH_GENERIC_CONTEXT_COROUTINES", "generic_coroutines"),
             self.define("BOOST_ROOT", spec["boost"].prefix),
             self.define("HWLOC_ROOT", spec["hwloc"].prefix),


Test patch: