Repository: easybuilders/easybuild-easyblocks

Issue:
Regression: Broken templating in EasyBuild 4.7.0
I am not sure exactly when it happened, but recent EasyBuild versions broke template replacement in `configopts` when using `multi_deps`, i.e. : 
```python
multi_deps = {'Python': ['3.10', '3.9', '3.8'] }
configopts = "-DPYQT5_SIP_DIR=$EBROOTQT5/share/python%(pyshortver)s/site-packages/sip/PyQt5 -DPYQT5_MOD_DIR=$EBROOTQT5/lib/python%(pyshortver)s/site-packages/PyQt5 "
``` 
for a `CMakeMake` EasyBlock (maybe others, I have not tested) no longer yield builds with different `configopts` argument based on `%(pyshortver)s`. 

In a previous build log, `%(pyshortver)s` gets replaced for every iteration: 
```
grep run.py /cvmfs/soft.computecanada.ca/easybuild/software/2020/avx2/Compiler/gcc9/qgis/3.16.10/easybuild/*.log | grep cmake | grep running
== 2021-08-31 19:14:35,595 run.py:233 INFO running cmd:  cmake -DCMAKE_INSTALL_PREFIX=/cvmfs/soft.computecanada.ca/easybuild/software/2020/avx2/Compiler/gcc9/qgis/3.16.10 -DCMAKE_BUILD_TYPE=Release -DCMAKE_POSITION_INDEPENDENT_CODE=ON -DCMAKE_C_COMPILER='gcc' -DCMAKE_C_FLAGS='-O2 -ftree-vectorize -march=core-avx2 -fno-math-errno -fPIC -std=gnu++11' -DCMAKE_CXX_COMPILER='g++' -DCMAKE_CXX_FLAGS='-O2 -ftree-vectorize -march=core-avx2 -fno-math-errno -fPIC -std=gnu++11' -DCMAKE_Fortran_COMPILER='gfortran' -DCMAKE_Fortran_FLAGS='-O2 -ftree-vectorize -march=core-avx2 -fno-math-errno -fPIC' -DCMAKE_VERBOSE_MAKEFILE=ON -DCMAKE_FIND_USE_PACKAGE_REGISTRY=FALSE  -DCMAKE_VERBOSE_MAKEFILE:BOOL=ON  -DENABLE_XHOST=OFF  -DCMAKE_SKIP_INSTALL_RPATH=ON -DCMAKE_PREFIX_PATH=$EBROOTQT -DQJSON_DIR=$EBROOTQJSON/lib/cmake/qjson -DQSCINTILLA_LIBRARY=$EBROOTQSCINTILLA/lib/libqscintilla2_qt5.so -DWITH_QTWEBKIT=OFF -DPYQT5_SIP_DIR=$EBROOTQT5/share/python3.7/sip/PyQt5 -DQSCI_SIP_DIR=$EBROOTQSCINTILLA/share/sip/PyQt5 /tmp/ebuser/avx2/QGIS/3.16.10/GCC-9.3.0/qgis-3.16.10/
== 2021-08-31 22:14:21,025 run.py:233 INFO running cmd:  cmake -DCMAKE_INSTALL_PREFIX=/cvmfs/soft.computecanada.ca/easybuild/software/2020/avx2/Compiler/gcc9/qgis/3.16.10 -DCMAKE_BUILD_TYPE=Release -DCMAKE_POSITION_INDEPENDENT_CODE=ON -DCMAKE_C_COMPILER='gcc' -DCMAKE_C_FLAGS='-O2 -ftree-vectorize -march=core-avx2 -fno-math-errno -fPIC -std=gnu++11' -DCMAKE_CXX_COMPILER='g++' -DCMAKE_CXX_FLAGS='-O2 -ftree-vectorize -march=core-avx2 -fno-math-errno -fPIC -std=gnu++11' -DCMAKE_Fortran_COMPILER='gfortran' -DCMAKE_Fortran_FLAGS='-O2 -ftree-vectorize -march=core-avx2 -fno-math-errno -fPIC' -DCMAKE_VERBOSE_MAKEFILE=ON -DCMAKE_FIND_USE_PACKAGE_REGISTRY=FALSE  -DCMAKE_VERBOSE_MAKEFILE:BOOL=ON  -DENABLE_XHOST=OFF  -DCMAKE_SKIP_INSTALL_RPATH=ON -DCMAKE_PREFIX_PATH=$EBROOTQT -DQJSON_DIR=$EBROOTQJSON/lib/cmake/qjson -DQSCINTILLA_LIBRARY=$EBROOTQSCINTILLA/lib/libqscintilla2_qt5.so -DWITH_QTWEBKIT=OFF -DPYQT5_SIP_DIR=$EBROOTQT5/share/python3.8/sip/PyQt5 -DQSCI_SIP_DIR=$EBROOTQSCINTILLA/share/sip/PyQt5 /tmp/ebuser/avx2/QGIS/3.16.10/GCC-9.3.0/qgis-3.16.10/
== 2021-08-31 23:54:14,144 run.py:233 INFO running cmd:  cmake -DCMAKE_INSTALL_PREFIX=/cvmfs/soft.computecanada.ca/easybuild/software/2020/avx2/Compiler/gcc9/qgis/3.16.10 -DCMAKE_BUILD_TYPE=Release -DCMAKE_POSITION_INDEPENDENT_CODE=ON -DCMAKE_C_COMPILER='gcc' -DCMAKE_C_FLAGS='-O2 -ftree-vectorize -march=core-avx2 -fno-math-errno -fPIC -std=gnu++11' -DCMAKE_CXX_COMPILER='g++' -DCMAKE_CXX_FLAGS='-O2 -ftree-vectorize -march=core-avx2 -fno-math-errno -fPIC -std=gnu++11' -DCMAKE_Fortran_COMPILER='gfortran' -DCMAKE_Fortran_FLAGS='-O2 -ftree-vectorize -march=core-avx2 -fno-math-errno -fPIC' -DCMAKE_VERBOSE_MAKEFILE=ON -DCMAKE_FIND_USE_PACKAGE_REGISTRY=FALSE  -DCMAKE_VERBOSE_MAKEFILE:BOOL=ON  -DENABLE_XHOST=OFF  -DCMAKE_SKIP_INSTALL_RPATH=ON -DCMAKE_PREFIX_PATH=$EBROOTQT -DQJSON_DIR=$EBROOTQJSON/lib/cmake/qjson -DQSCINTILLA_LIBRARY=$EBROOTQSCINTILLA/lib/libqscintilla2_qt5.so -DWITH_QTWEBKIT=OFF -DPYQT5_SIP_DIR=$EBROOTQT5/share/python3.9/sip/PyQt5 -DQSCI_SIP_DIR=$EBROOTQSCINTILLA/share/sip/PyQt5 /tmp/ebuser/avx2/QGIS/3.16.10/GCC-9.3.0/qgis-3.16.10/
``` 

while in a recent build, it keeps the same value as for the first iteration: 
``` 
$ grep run.py /tmp/eb-3yw8sr68/easybuild-6ry1d4vr.log | grep cmake | grep running
== 2023-02-08 16:18:35,282 run.py:236 INFO running cmd: patch -p1 -i /home/mboisson/git/easybuild-easyconfigs/easybuild/easyconfigs/q/QGIS/QGIS-3.22.14-FindPyQt5.cmake.patch
== 2023-02-08 16:18:56,589 run.py:236 INFO running cmd:  cmake  -DCMAKE_INSTALL_PREFIX=/home/mboisson/.local/easybuild/software/2020/avx2/Compiler/gcc9/qgis/3.22.14 -DCMAKE_BUILD_TYPE=Release -DCMAKE_POSITION_INDEPENDENT_CODE=ON -DCMAKE_VERBOSE_MAKEFILE=ON -DCMAKE_FIND_USE_PACKAGE_REGISTRY=OFF  -DCMAKE_VERBOSE_MAKEFILE:BOOL=ON  -DENABLE_XHOST=OFF  -DCMAKE_SKIP_INSTALL_RPATH=ON -DWITH_QTWEBKIT=OFF -DPYQT5_SIP_DIR=$EBROOTQT5/share/python3.10/site-packages/sip/PyQt5 -DPYQT5_MOD_DIR=$EBROOTQT5/lib/python3.10/site-packages/PyQt5 -DQSCI_SIP_DIR=$EBROOTQSCINTILLA/share/sip/PyQt5 -DPYQT5_VERSION_STR=5.15.5  /tmp/mboisson/avx2/QGIS/3.22.14/GCC-9.3.0/qgis-3.22.14/
== 2023-02-08 16:31:47,766 run.py:236 INFO running cmd: patch -p1 -i /home/mboisson/git/easybuild-easyconfigs/easybuild/easyconfigs/q/QGIS/QGIS-3.22.14-FindPyQt5.cmake.patch
== 2023-02-08 16:32:07,346 run.py:236 INFO running cmd:  cmake   -DCMAKE_INSTALL_PREFIX=/home/mboisson/.local/easybuild/software/2020/avx2/Compiler/gcc9/qgis/3.22.14 -DCMAKE_BUILD_TYPE=Release -DCMAKE_POSITION_INDEPENDENT_CODE=ON -DCMAKE_VERBOSE_MAKEFILE=ON -DCMAKE_FIND_USE_PACKAGE_REGISTRY=OFF  -DCMAKE_VERBOSE_MAKEFILE:BOOL=ON  -DENABLE_XHOST=OFF  -DCMAKE_SKIP_INSTALL_RPATH=ON -DWITH_QTWEBKIT=OFF -DPYQT5_SIP_DIR=$EBROOTQT5/share/python3.10/site-packages/sip/PyQt5 -DPYQT5_MOD_DIR=$EBROOTQT5/lib/python3.10/site-packages/PyQt5 -DQSCI_SIP_DIR=$EBROOTQSCINTILLA/share/sip/PyQt5 -DPYQT5_VERSION_STR=5.15.5  /tmp/mboisson/avx2/QGIS/3.22.14/GCC-9.3.0/qgis-3.22.14/
``` 


Discussion:
According to this, it worked at least up to version 4.6.0: 
```
for f in $(grep -rl "configopts.*pyshortver" /cvmfs/soft.computecanada.ca/easybuild/ebfiles_repo/2020/); do grep easybuild-framework_version $f; done
    "easybuild-framework_version": "4.4.1-r73a32642a48430484a7c9a4654e7afda27268be4",
    "easybuild-framework_version": "4.3.3-r15719a0ece2011cea9ce176d1763cdc620b188f4",
    "easybuild-framework_version": "4.6.0-ra6dbb4b22832c431590f725219d89d82c9946dba",
    "easybuild-framework_version": "4.6.0-ra6dbb4b22832c431590f725219d89d82c9946dba",
    "easybuild-framework_version": "4.6.0-ra6dbb4b22832c431590f725219d89d82c9946dba",
    "easybuild-framework_version": "4.6.0-ra6dbb4b22832c431590f725219d89d82c9946dba",
    "easybuild-framework_version": "4.3.3-r3ff498d1f735c51b313c04c5e4abeeaecc23589b",
    "easybuild-framework_version": "4.3.3-r3ff498d1f735c51b313c04c5e4abeeaecc23589b",
    "easybuild-framework_version": "4.2.2-r64415f53d9bdff498788ce494f68a98641a3883d",
    "easybuild-framework_version": "4.2.2-r64415f53d9bdff498788ce494f68a98641a3883d",
    "easybuild-framework_version": "4.4.0-r5824ede87a67bf66b912bcb3f7d73c27835716af",
    "easybuild-framework_version": "4.4.0-r5824ede87a67bf66b912bcb3f7d73c27835716af",
    "easybuild-framework_version": "4.4.0-r5824ede87a67bf66b912bcb3f7d73c27835716af",
    "easybuild-framework_version": "4.3.3-r15719a0ece2011cea9ce176d1763cdc620b188f4",
``` 

Looking at one of these recipes, built with EasyBuild 4.6.0: 
```
grep "run.py" /cvmfs/soft.computecanada.ca/easybuild/software/2020/avx2/MPI/gcc9/openmpi4/hoomd-blue-mpi/3.6.0/easybuild/easybuild-HOOMD-blue-3.6.0-20221117.183917.log | grep cmake | grep running
== 2022-11-17 18:03:15,250 run.py:233 INFO running cmd:  cmake -DCMAKE_INSTALL_PREFIX=/cvmfs/soft.computecanada.ca/easybuild/software/2020/avx2/MPI/gcc9/openmpi4/hoomd-blue-mpi/3.6.0 -DCMAKE_BUILD_TYPE=Release -DCMAKE_C_COMPILER='mpicc' -DCMAKE_C_FLAGS='-O2 -ftree-vectorize -march=core-avx2 -fno-math-errno' -DCMAKE_CXX_COMPILER='mpicxx' -DCMAKE_CXX_FLAGS='-O2 -ftree-vectorize -march=core-avx2 -fno-math-errno' -DCMAKE_Fortran_COMPILER='mpifort' -DCMAKE_Fortran_FLAGS='-O2 -ftree-vectorize -march=core-avx2 -fno-math-errno' -DCMAKE_VERBOSE_MAKEFILE=ON -DCMAKE_FIND_USE_PACKAGE_REGISTRY=FALSE  -DCMAKE_VERBOSE_MAKEFILE:BOOL=ON  -DENABLE_XHOST=OFF  -DCMAKE_SKIP_INSTALL_RPATH=ON -DENABLE_MPI=ON -DENABLE_CUDA=OFF -DENABLE_TBB=ON -DENABLE_LLVM=OFF -DCMAKE_INSTALL_PREFIX=/cvmfs/soft.computecanada.ca/easybuild/software/2020/avx2/MPI/gcc9/openmpi4/hoomd-blue-mpi/3.6.0/lib/python3.8/site-packages  /tmp/ebuser/avx2/HOOMDblue/3.6.0/gompi-2020a-mpi/hoomd-v3.6.0/
== 2022-11-17 18:15:51,643 run.py:233 INFO running cmd:  cmake -DCMAKE_INSTALL_PREFIX=/cvmfs/soft.computecanada.ca/easybuild/software/2020/avx2/MPI/gcc9/openmpi4/hoomd-blue-mpi/3.6.0 -DCMAKE_BUILD_TYPE=Release -DCMAKE_C_COMPILER='mpicc' -DCMAKE_C_FLAGS='-O2 -ftree-vectorize -march=core-avx2 -fno-math-errno' -DCMAKE_CXX_COMPILER='mpicxx' -DCMAKE_CXX_FLAGS='-O2 -ftree-vectorize -march=core-avx2 -fno-math-errno' -DCMAKE_Fortran_COMPILER='mpifort' -DCMAKE_Fortran_FLAGS='-O2 -ftree-vectorize -march=core-avx2 -fno-math-errno' -DCMAKE_VERBOSE_MAKEFILE=ON -DCMAKE_FIND_USE_PACKAGE_REGISTRY=FALSE  -DCMAKE_VERBOSE_MAKEFILE:BOOL=ON  -DENABLE_XHOST=OFF  -DCMAKE_SKIP_INSTALL_RPATH=ON -DENABLE_MPI=ON -DENABLE_CUDA=OFF -DENABLE_TBB=ON -DENABLE_LLVM=OFF -DCMAKE_INSTALL_PREFIX=/cvmfs/soft.computecanada.ca/easybuild/software/2020/avx2/MPI/gcc9/openmpi4/hoomd-blue-mpi/3.6.0/lib/python3.9/site-packages  /tmp/ebuser/avx2/HOOMDblue/3.6.0/gompi-2020a-mpi/hoomd-v3.6.0/
== 2022-11-17 18:27:07,729 run.py:233 INFO running cmd:  cmake -DCMAKE_INSTALL_PREFIX=/cvmfs/soft.computecanada.ca/easybuild/software/2020/avx2/MPI/gcc9/openmpi4/hoomd-blue-mpi/3.6.0 -DCMAKE_BUILD_TYPE=Release -DCMAKE_C_COMPILER='mpicc' -DCMAKE_C_FLAGS='-O2 -ftree-vectorize -march=core-avx2 -fno-math-errno' -DCMAKE_CXX_COMPILER='mpicxx' -DCMAKE_CXX_FLAGS='-O2 -ftree-vectorize -march=core-avx2 -fno-math-errno' -DCMAKE_Fortran_COMPILER='mpifort' -DCMAKE_Fortran_FLAGS='-O2 -ftree-vectorize -march=core-avx2 -fno-math-errno' -DCMAKE_VERBOSE_MAKEFILE=ON -DCMAKE_FIND_USE_PACKAGE_REGISTRY=FALSE  -DCMAKE_VERBOSE_MAKEFILE:BOOL=ON  -DENABLE_XHOST=OFF  -DCMAKE_SKIP_INSTALL_RPATH=ON -DENABLE_MPI=ON -DENABLE_CUDA=OFF -DENABLE_TBB=ON -DENABLE_LLVM=OFF -DCMAKE_INSTALL_PREFIX=/cvmfs/soft.computecanada.ca/easybuild/software/2020/avx2/MPI/gcc9/openmpi4/hoomd-blue-mpi/3.6.0/lib/python3.10/site-packages  /tmp/ebuser/avx2/HOOMDblue/3.6.0/gompi-2020a-mpi/hoomd-v3.6.0/
``` 
This commit from @Flamefire is my first suspect. It messes with `configopts` and has no explicit code to handle templates

https://github.com/easybuilders/easybuild-easyblocks/commit/0b78b0280ebbd8978110ce3853e265e515425a40
If I've got my testing correct:
* This broke between 4.6.0 and 4.6.1
* This impacts `CMake` only - not `ConfigureMake`
> If I've got my testing correct:
> 
> * This broke between 4.6.0 and 4.6.1
> * This impacts `CMake` only - not `ConfigureMake`

That would match the above commit, I think 
I confirmed that this can not be reproduced using `MEGAHIT-1.2.9-GCCcore-9.3.0.eb` (which uses `multi_deps` + `CMakeMake`) when fiddling with `preconfigopts`, so looks specific to `CMakeMake` and `configopts`.

[This mean](https://github.com/easybuilders/easybuild-easyblocks/commit/0b78b0280ebbd8978110ce3853e265e515425a40) is indeed a likely culprit, so I'll move this issue into the easyblocks repo...
There's another recently opened issue (https://github.com/easybuilders/easybuild-easyconfigs/issues/17259) caused by this PR https://github.com/easybuilders/easybuild-easyblocks/pull/2514



Resolving patch:
diff --git a/easybuild/easyblocks/generic/cmakemake.py b/easybuild/easyblocks/generic/cmakemake.py
--- a/easybuild/easyblocks/generic/cmakemake.py
+++ b/easybuild/easyblocks/generic/cmakemake.py
@@ -141,7 +141,10 @@ def build_type(self):
 
     def prepend_config_opts(self, config_opts):
         """Prepends configure options (-Dkey=value) to configopts ignoring those already set"""
-        cfg_configopts = self.cfg['configopts']
+        # need to disable template resolution or it will remain the same for all runs
+        with self.cfg.disable_templating():
+            cfg_configopts = self.cfg['configopts']
+
         # All options are of the form '-D<key>=<value>'
         new_opts = ' '.join('-D%s=%s' % (key, value) for key, value in config_opts.items()
                             if '-D%s=' % key not in cfg_configopts)


Test patch: