internlm/SWE-Fixer-Train-110K huggingface dataset:
Repository: easybuilders/easybuild-easyblocks

Issue:
_should_unpack_source in PythonPackage breaks if there are no sources
https://github.com/easybuilders/easybuild-easyblocks/blob/2c20ad0899067d91c4ac05aade254903249a075c/easybuild/easyblocks/generic/pythonpackage.py#L425-L434

assumes there is at least one source, either a string or a list. But e.g. MATLAB-Engine doesn't use its own sources at all so these are now broken.

Simple fix we check is there are no sources, then `return False` 


Discussion:

Resolving patch:
diff --git a/easybuild/easyblocks/generic/pythonpackage.py b/easybuild/easyblocks/generic/pythonpackage.py
--- a/easybuild/easyblocks/generic/pythonpackage.py
+++ b/easybuild/easyblocks/generic/pythonpackage.py
@@ -424,13 +424,27 @@ def prepare_python(self):
 
     def _should_unpack_source(self):
         """Determine whether we need to unpack the source(s)"""
+
         unpack_sources = self.cfg.get('unpack_sources')
+
+        # if unpack_sources is not specified, check file extension of (first) source file
         if unpack_sources is None:
             src = self.src
-            if not isinstance(src, string_type):
-                src = src[0]['path']
-            _, ext = os.path.splitext(src)
-            unpack_sources = ext.lower() != '.whl'
+            # we may have a list of sources, only consider first source file in that case
+            if isinstance(src, (list, tuple)):
+                if src:
+                    src = src[0]
+                    # source file specs (incl. path) could be specified via a dict
+                    if isinstance(src, dict) and 'path' in src:
+                        src = src['path']
+                else:
+                    unpack_sources = False
+
+            # if undecided, check the source file extension: don't try to unpack wheels (*.whl)
+            if unpack_sources is None:
+                _, ext = os.path.splitext(src)
+                unpack_sources = ext.lower() != '.whl'
+
         return unpack_sources
 
     def get_installed_python_packages(self, names_only=True, python_cmd=None):


Test patch:
Repository: easybuilders/easybuild-easyblocks

Issue:
crash when using --module-only with easyconfig using PythonBundle
```
$ eb antiSMASH-5.1.2-foss-2019b-Python-3.7.4.eb --module-only
ERROR: Traceback (most recent call last):
  File "/lib/python2.7/site-packages/easybuild_framework-4.3.5.dev0-py2.7.egg/easybuild/main.py", line 117, in build_and_install_software
    (ec_res['success'], app_log, err) = build_and_install_one(ec, init_env)
  File "/lib/python2.7/site-packages/easybuild_framework-4.3.5.dev0-py2.7.egg/easybuild/framework/easyblock.py", line 3633, in build_and_install_one
    result = app.run_all_steps(run_test_cases=run_test_cases)
  File "/lib/python2.7/site-packages/easybuild_framework-4.3.5.dev0-py2.7.egg/easybuild/framework/easyblock.py", line 3531, in run_all_steps
    self.run_step(step_name, step_methods)
  File "/lib/python2.7/site-packages/easybuild_framework-4.3.5.dev0-py2.7.egg/easybuild/framework/easyblock.py", line 3386, in run_step
    step_method(self)()
  File "/lib/python2.7/site-packages/easybuild_easyblocks-4.3.5.dev0-py2.7.egg/easybuild/easyblocks/generic/pythonbundle.py", line 173, in sanity_check_step
    super(Bundle, self).sanity_check_step(*args, **kwargs)
  File "/llib/python2.7/site-packages/easybuild_framework-4.3.5.dev0-py2.7.egg/easybuild/framework/easyblock.py", line 2524, in sanity_check_step
    self._sanity_check_step(*args, **kwargs)
  File "/lib/python2.7/site-packages/easybuild_framework-4.3.5.dev0-py2.7.egg/easybuild/framework/easyblock.py", line 3053, in _sanity_check_step
    self._sanity_check_step_extensions()
  File "/lib/python2.7/site-packages/easybuild_framework-4.3.5.dev0-py2.7.egg/easybuild/framework/easyblock.py", line 2926, in _sanity_check_step_extensions
    res = ext.sanity_check_step()
  File "/data/gent/400/vsc40023/easybuild_easy_installed/lib/python2.7/site-packages/easybuild_easyblocks-4.3.5.dev0-py2.7.egg/easybuild/easyblocks/generic/pythonpackage.py", line 822, in sanity_check_step
    exts_filter = (orig_exts_filter[0].replace('python', self.python_cmd), orig_exts_filter[1])
TypeError: replace() argument 2 must be str, not None
```


Discussion:
That's a side effect of now also running the sanity check for extensions under `--module-only`, see https://github.com/easybuilders/easybuild-framework/pull/3655
`eb Python-3.9.5-GCCcore-10.3.0.eb --module-only` is broken in the same way, so this is worth fixing ASAP...

Resolving patch:
diff --git a/easybuild/easyblocks/generic/pythonpackage.py b/easybuild/easyblocks/generic/pythonpackage.py
--- a/easybuild/easyblocks/generic/pythonpackage.py
+++ b/easybuild/easyblocks/generic/pythonpackage.py
@@ -816,6 +816,8 @@ def sanity_check_step(self, *args, **kwargs):
         else:
             # 'python' is replaced by full path to active 'python' command
             # (which is required especially when installing with system Python)
+            if self.python_cmd is None:
+                self.prepare_python()
             python_cmd = self.python_cmd
             if 'exts_filter' not in kwargs:
                 orig_exts_filter = EXTS_FILTER_PYTHON_PACKAGES


Test patch:
Repository: easybuilders/easybuild-easyblocks

Issue:
--module-only does not respect altroot
In `CUDA-11.1.1-GCC-10.2.0.eb` we have `altroot = 'CUDAcore'`. When I build the module (`eb CUDA-11.1.1-GCC-10.2.0.eb -Tr --rebuild`) I get:
```
setenv("EBROOTCUDA", "/rds/bear-apps/devel/eb-sjb-up/EL8/EL8-has/software/CUDAcore/11.1.1")
```
but when I add ` --module-only` this is replaced with:
```
setenv("EBROOTCUDA", root)
```
and `root` is set to `/rds/bear-apps/devel/eb-sjb-up/EL8/EL8-has/software/CUDA/11.1.1-GCC-10.2.0`.

This then breaks any software that uses `get_software_root('CUDA')` to determine the location of the CUDA files. For example, TensorFlow which fails to find `cublas_api.h` and so aborts the install.


Discussion:
This is an issue in the Bundle easyblock. `self.altroot` and `self.altversion` are set in `configure_step`, but this is not run in the `--module-only` case.

This also impacts GCC, which also uses `altroot`.

Resolving patch:
diff --git a/easybuild/easyblocks/generic/bundle.py b/easybuild/easyblocks/generic/bundle.py
--- a/easybuild/easyblocks/generic/bundle.py
+++ b/easybuild/easyblocks/generic/bundle.py
@@ -180,15 +180,19 @@ def patch_step(self):
         """Patch step must be a no-op for bundle, since there are no top-level sources/patches."""
         pass
 
-    def configure_step(self):
-        """Collect altroot/altversion info."""
-        # pick up altroot/altversion, if they are defined
-        self.altroot = None
+    def get_altroot_and_altversion(self):
+        """Get altroot and altversion, if they are defined"""
+        altroot = None
         if self.cfg['altroot']:
-            self.altroot = get_software_root(self.cfg['altroot'])
-        self.altversion = None
+            altroot = get_software_root(self.cfg['altroot'])
+        altversion = None
         if self.cfg['altversion']:
-            self.altversion = get_software_version(self.cfg['altversion'])
+            altversion = get_software_version(self.cfg['altversion'])
+        return altroot, altversion
+
+    def configure_step(self):
+        """Collect altroot/altversion info."""
+        self.altroot, self.altversion = self.get_altroot_and_altversion()
 
     def build_step(self):
         """Do nothing."""
@@ -269,6 +273,9 @@ def install_step(self):
 
     def make_module_extra(self, *args, **kwargs):
         """Set extra stuff in module file, e.g. $EBROOT*, $EBVERSION*, etc."""
+        if not self.altroot and not self.altversion:
+            # check for altroot and altversion (needed here for a module only build)
+            self.altroot, self.altversion = self.get_altroot_and_altversion()
         if 'altroot' not in kwargs:
             kwargs['altroot'] = self.altroot
         if 'altversion' not in kwargs:


Test patch:
Repository: easybuilders/easybuild-easyblocks

Issue:
ESMF sanity check
Hi, There is a newer release version of EMSF available 8.1.1.
The current `sanity_check_step`  in `/easybuild/easyblocks/e/esmf.py`  (line 120 or so) will not work since the filenames for the binaries have been changed in 8.1.1 (the current `esmf.py` checks for `ESMF_Info` but since the 8.1.0 it's acutally `EMSF_PrintInfo`, same with `ESMF_InfoC` and `ESMF_PrintInfoC`). 

Also, it seems that the `ESMF-6.1.1_libopts.patch` is no longer needed in the `easyconfig`

And source url + the naming scheme have changed:
```
source_urls = ['https://github.com/esmf-org/esmf/archive/refs/tags/']
sources = ['%%(name)s_%s.tar.gz' % '_'.join(version.split('.'))]
```



Discussion:

Resolving patch:
diff --git a/easybuild/easyblocks/e/esmf.py b/easybuild/easyblocks/e/esmf.py
--- a/easybuild/easyblocks/e/esmf.py
+++ b/easybuild/easyblocks/e/esmf.py
@@ -117,7 +117,13 @@ def configure_step(self):
     def sanity_check_step(self):
         """Custom sanity check for ESMF."""
 
-        binaries = ['ESMF_Info', 'ESMF_InfoC', 'ESMF_RegridWeightGen', 'ESMF_WebServController']
+        if LooseVersion(self.version) < LooseVersion('8.1.0'):
+            binaries = ['ESMF_Info', 'ESMF_InfoC', 'ESMF_Regrid', 'ESMF_RegridWeightGen',
+                        'ESMF_Scrip2Unstruct', 'ESMF_WebServController']
+        else:
+            binaries = ['ESMF_PrintInfo', 'ESMF_PrintInfoC', 'ESMF_Regrid', 'ESMF_RegridWeightGen',
+                        'ESMF_Scrip2Unstruct', 'ESMF_WebServController']
+
         libs = ['libesmf.a', 'libesmf.%s' % get_shared_lib_ext()]
         custom_paths = {
             'files': [os.path.join('bin', x) for x in binaries] + [os.path.join('lib', x) for x in libs],


Test patch:
Repository: easybuilders/easybuild-easyblocks

Issue:
Clang builds fail if a non-CUDA version is built in the same session after a CUDA version
We've seen this in https://github.com/easybuilders/easybuild-easyconfigs/pull/13746

The build failure for non-CUDA versions was
```
Can't build Clang with CUDA support without specifying 'cuda-compute-capabilities' (at easybuild/easybuild-easyblocks/easybuild/easyblocks/c/clang.py:327 in configure_step)
```
but that should only be a possible error if we're doing a CUDA build.

The issue is that we're modifying the lists in `DEFAULT_TARGETS_MAP`:

```
>>> DEFAULT_TARGETS_MAP = { 'POWER': ['PowerPC'], 'X86_64': ['X86'] }
>>> a = DEFAULT_TARGETS_MAP.get('X86_64', None)
>>> a += ['NVPTX']
>>> DEFAULT_TARGETS_MAP
{'POWER': ['PowerPC'], 'X86_64': ['X86', 'NVPTX']}
```

We have similar lookups in `llvm.py` but there we do not add to the list, so (currently) there is no issue.

In `lammps.py` the items are strings, which are immutable.


Discussion:

Resolving patch:
diff --git a/easybuild/easyblocks/c/clang.py b/easybuild/easyblocks/c/clang.py
--- a/easybuild/easyblocks/c/clang.py
+++ b/easybuild/easyblocks/c/clang.py
@@ -193,14 +193,14 @@ def prepare_step(self, *args, **kwargs):
         build_targets = self.cfg['build_targets']
         if build_targets is None:
             arch = get_cpu_architecture()
-            default_targets = DEFAULT_TARGETS_MAP.get(arch, None)
-            if default_targets:
+            try:
+                default_targets = DEFAULT_TARGETS_MAP[arch][:]
                 # If CUDA is included as a dep, add NVPTX as a target (could also support AMDGPU if we knew how)
                 if get_software_root("CUDA"):
                     default_targets += ["NVPTX"]
                 self.cfg['build_targets'] = build_targets = default_targets
                 self.log.debug("Using %s as default build targets for CPU/GPU architecture %s.", default_targets, arch)
-            else:
+            except KeyError:
                 raise EasyBuildError("No default build targets defined for CPU architecture %s.", arch)
 
         # carry on with empty list from this point forward if no build targets are specified
diff --git a/easybuild/easyblocks/l/llvm.py b/easybuild/easyblocks/l/llvm.py
--- a/easybuild/easyblocks/l/llvm.py
+++ b/easybuild/easyblocks/l/llvm.py
@@ -65,11 +65,11 @@ def configure_step(self):
         build_targets = self.cfg['build_targets']
         if build_targets is None:
             arch = get_cpu_architecture()
-            default_targets = DEFAULT_TARGETS_MAP.get(arch, None)
-            if default_targets:
+            try:
+                default_targets = DEFAULT_TARGETS_MAP[arch][:]
                 self.cfg['build_targets'] = build_targets = default_targets
                 self.log.debug("Using %s as default build targets for CPU architecture %s.", default_targets, arch)
-            else:
+            except KeyError:
                 raise EasyBuildError("No default build targets defined for CPU architecture %s.", arch)
 
         unknown_targets = [target for target in build_targets if target not in CLANG_TARGETS]


Test patch:
Repository: easybuilders/easybuild-easyblocks

Issue:
GTK+ check in OpenCV easyblock
https://github.com/easybuilders/easybuild-easyconfigs/pull/13900 and https://github.com/easybuilders/easybuild-easyconfigs/pull/13893 mean that newer `GTK+` are now either `GTK2` and `GTK3`. The OpenCV block checks (https://github.com/easybuilders/easybuild-easyblocks/blob/develop/easybuild/easyblocks/o/opencv.py#L152) for `GTK+` and sets `-DWITH_GTK=OFF` if this is not found. The check will need updating.

I do not know if OpenCV can build with both GTK2 and GTK3 at the same time.


Discussion:
> I do not know if OpenCV can build with both GTK2 and GTK3 at the same time.

OK, I've checked OpenCV and it builds against one of GTK2 or GTK3 but not both at the same time.

Resolving patch:
diff --git a/easybuild/easyblocks/o/opencv.py b/easybuild/easyblocks/o/opencv.py
--- a/easybuild/easyblocks/o/opencv.py
+++ b/easybuild/easyblocks/o/opencv.py
@@ -26,6 +26,7 @@
 EasyBuild support for building and installing OpenCV, implemented as an easyblock
 
 @author: Kenneth Hoste (Ghent University)
+@author: Simon Branford (University of Birmingham)
 """
 import glob
 import os
@@ -153,6 +154,10 @@ def configure_step(self):
         if get_software_root('GTK+'):
             if LooseVersion(get_software_version('GTK+')) < LooseVersion('3.0'):
                 self.cfg.update('configopts', '-DWITH_GTK_2_X=ON')
+        elif get_software_root('GTK3'):
+            pass
+        elif get_software_root('GTK2'):
+            self.cfg.update('configopts', '-DWITH_GTK_2_X=ON')
         else:
             self.cfg.update('configopts', '-DWITH_GTK=OFF')
 


Test patch:
Repository: easybuilders/easybuild-easyblocks

Issue:
TensorFlow 1 coexist with gcc nvptx
Line 680 in tensorflow.py assumes there is only one directory matching glob gcc_root/lib/gcc/*

This fails for the case of GCC 10.2.0 for example because nvptx is also a thing that exists. 

This has never been an issue before because no one has tried to do TensorFlow 1 + newer toolchains before. The mad man! 

I think probably the glob should specify a linux variant or exclude otherwise exclude nvptx. So far all the examples I have seen in the wild look like x86_64-pc-linux-gnu and TensorFlow 1 itself is hardcoded to look for directories that look like that. I don't think we want the nvptx. I am not aware of what else could appear in that directory. 



Discussion:

Resolving patch:
diff --git a/easybuild/easyblocks/t/tensorflow.py b/easybuild/easyblocks/t/tensorflow.py
--- a/easybuild/easyblocks/t/tensorflow.py
+++ b/easybuild/easyblocks/t/tensorflow.py
@@ -672,9 +672,10 @@ def patch_crosstool_files(self):
             gcc_ver = get_software_version('GCCcore') or get_software_version('GCC')
 
             # figure out location of GCC include files
-            res = glob.glob(os.path.join(gcc_root, 'lib', 'gcc', '*', gcc_ver, 'include'))
+            # make sure we don't pick up the nvptx-none directory by looking for a specific include file
+            res = glob.glob(os.path.join(gcc_root, 'lib', 'gcc', '*', gcc_ver, 'include', 'immintrin.h'))
             if res and len(res) == 1:
-                gcc_lib_inc = res[0]
+                gcc_lib_inc = os.path.dirname(res[0])
                 inc_paths.append(gcc_lib_inc)
             else:
                 raise EasyBuildError("Failed to pinpoint location of GCC include files: %s", res)


Test patch:
Repository: easybuilders/easybuild-easyblocks

Issue:
intel-compilers modules don't set MANPATH
With the newer intel-compilers modules (and thus in the intel toolchain starting from 2021a I guess), the MANPATH variable is not set. The IntelBase block seems to look for a man/ subfolder only:

https://github.com/easybuilders/easybuild-easyblocks/blob/develop/easybuild/easyblocks/generic/intelbase.py#L124

while in the newer installations, the man pages are in a different place, e.g. 

intel-compilers/2021.1.2/compiler/2021.1.2/documentation/en/man/common/


Discussion:

Resolving patch:
diff --git a/easybuild/easyblocks/i/intel_compilers.py b/easybuild/easyblocks/i/intel_compilers.py
--- a/easybuild/easyblocks/i/intel_compilers.py
+++ b/easybuild/easyblocks/i/intel_compilers.py
@@ -147,6 +147,9 @@ def make_module_req_guess(self):
             ],
             'LD_LIBRARY_PATH': libdirs,
             'LIBRARY_PATH': libdirs,
+            'MANPATH': [
+                os.path.join('compiler', self.version, 'documentation', 'en', 'man', 'common'),
+            ],
             'OCL_ICD_FILENAMES': [
                 os.path.join(self.compilers_subdir, 'lib', 'x64', 'libintelocl.so'),
             ],


Test patch:
Repository: easybuilders/easybuild-easyblocks

Issue:
Revisit default use of `ld.gold` in binutils easyblock
Someone mentioned in Slack that `ld.gold` is no longer being actively developed, so we should probably switch (back) to `ld.bfd` as the default. 


Discussion:
The question here is when we flip the default.
Only for future GCC versions (> 11.2.0)?
Only in EasyBuild v5.0?

There may be some fallout of making a switch like this. Maybe we need an EasyBuild configuration option to control this?

Resolving patch:
diff --git a/easybuild/easyblocks/g/gcc.py b/easybuild/easyblocks/g/gcc.py
--- a/easybuild/easyblocks/g/gcc.py
+++ b/easybuild/easyblocks/g/gcc.py
@@ -89,7 +89,8 @@ def extra_options():
             'pplwatchdog': [False, "Enable PPL watchdog", CUSTOM],
             'prefer_lib_subdir': [False, "Configure GCC to prefer 'lib' subdirs over 'lib64' when linking", CUSTOM],
             'profiled': [False, "Bootstrap GCC with profile-guided optimizations", CUSTOM],
-            'use_gold_linker': [True, "Configure GCC to use GOLD as default linker", CUSTOM],
+            'use_gold_linker': [None, "Configure GCC to use GOLD as default linker "
+                                      "(default: True for GCC < 11.3.0)", CUSTOM],
             'withcloog': [False, "Build GCC with CLooG support", CUSTOM],
             'withisl': [False, "Build GCC with ISL support", CUSTOM],
             'withlibiberty': [False, "Enable installing of libiberty", CUSTOM],
@@ -129,6 +130,10 @@ def __init__(self, *args, **kwargs):
             if version < LooseVersion('5.0.0') and self.cfg['withisl'] and not self.cfg['withcloog']:
                 raise EasyBuildError("Activating ISL without CLooG is pointless")
 
+            # Disable the Gold linker by default for GCC 11.3.0 and newer, as it suffers from bitrot
+            if self.cfg['use_gold_linker'] is None:
+                self.cfg['use_gold_linker'] = version < LooseVersion('11.3.0')
+
         # unset some environment variables that are known to may cause nasty build errors when bootstrapping
         self.cfg.update('unwanted_env_vars', ['CPATH', 'C_INCLUDE_PATH', 'CPLUS_INCLUDE_PATH', 'OBJC_INCLUDE_PATH'])
         # ubuntu needs the LIBRARY_PATH env var to work apparently (#363)


Test patch:
Repository: easybuilders/easybuild-easyblocks

Issue:
fix sanity checks for Clang 11 when built with NVPTX support
Clang creates the following `libomptarget*nvptx*.bc` according to its version (11, 12, 13). The current sanity check is handling wrongly 11.0.1 case.

```
# Clang 11.0.1
libomptarget-nvptx-sm_70.bc

libomptarget-nvptx.a
libomptarget.rtl.x86_64.so
libomptarget.so
libomptarget.rtl.cuda.so

# Clang 12.0.1
libomptarget-nvptx-cuda_80-sm_70.bc
libomptarget-nvptx-cuda_90-sm_70.bc
libomptarget-nvptx-cuda_91-sm_70.bc
libomptarget-nvptx-cuda_92-sm_70.bc
libomptarget-nvptx-cuda_100-sm_70.bc
libomptarget-nvptx-cuda_101-sm_70.bc
libomptarget-nvptx-cuda_102-sm_70.bc
libomptarget-nvptx-cuda_110-sm_70.bc
libomptarget-nvptx-cuda_111-sm_70.bc
libomptarget-nvptx-cuda_112-sm_70.bc

libomptarget.rtl.x86_64.so
libomptarget.so
libomptarget.rtl.cuda.so
libomptarget.so.12

# Clang 13.0.1
libomptarget-amdgcn-gfx700.bc
libomptarget-amdgcn-gfx701.bc
libomptarget-amdgcn-gfx801.bc
libomptarget-amdgcn-gfx803.bc
libomptarget-amdgcn-gfx900.bc
libomptarget-amdgcn-gfx902.bc
libomptarget-amdgcn-gfx906.bc
libomptarget-amdgcn-gfx908.bc

libomptarget-new-nvptx-sm_70.bc
libomptarget-nvptx-sm_70.bc

libomptarget.rtl.x86_64.so
libomptarget.so
libomptarget.rtl.cuda.so
libomptarget.rtl.amdgpu.so
```
I think the behavior when there is CUDA version included is typical only for v12 whereas in the current test it is applied on v11 too.

```Python
# clang.py (develop)

if LooseVersion('11.0') < LooseVersion(self.version) < LooseVersion('13.0'):
    custom_paths['files'].extend(["lib/libomptarget-nvptx-cuda_%s-sm_%s.bc" % (x, y)
                                 for x in CUDA_TOOLKIT_SUPPORT for y in cuda_cc])
else:
    custom_paths['files'].extend(["lib/libomptarget-nvptx-sm_%s.bc" % cc
                                 for cc in cuda_cc])
```

I was building on x86_64 with V100.


Discussion:

Resolving patch:
diff --git a/easybuild/easyblocks/c/clang.py b/easybuild/easyblocks/c/clang.py
--- a/easybuild/easyblocks/c/clang.py
+++ b/easybuild/easyblocks/c/clang.py
@@ -554,7 +554,7 @@ def sanity_check_step(self):
             cuda_cc = cfg_cuda_cc or ec_cuda_cc or []
             # We need the CUDA capability in the form of '75' and not '7.5'
             cuda_cc = [cc.replace('.', '') for cc in cuda_cc]
-            if LooseVersion('11.0') < LooseVersion(self.version) < LooseVersion('13.0'):
+            if LooseVersion('12.0') < LooseVersion(self.version) < LooseVersion('13.0'):
                 custom_paths['files'].extend(["lib/libomptarget-nvptx-cuda_%s-sm_%s.bc" % (x, y)
                                              for x in CUDA_TOOLKIT_SUPPORT for y in cuda_cc])
             else:


Test patch: