Repository: easybuilders/easybuild-framework

Issue:
eb --list-toolchains. Error parsing custom toolchains 
Using Easybuild 4.3.0 on Ubuntu 18.04 I have noticed this error message using `eb --list-toolchains` when custom toolchains are defined:

```
ERROR: Failed to parse configuration options: '<' not supported between instances of 'type' and 'type'
```

This bug does not influence the functionality of the defined toolchain, which is working fine. I can reproduce this error with a dummy toolchain. The error is triggered as soon as the **NAME** option is set in the class implementation:
```python
"""
EasyBuild support for a dummy test toolchain 
"""
from easybuild.toolchains.compiler.gcc import Gcc
from easybuild.tools.toolchain.toolchain import SYSTEM_TOOLCHAIN_NAME

class TestDummy(Gcc):
    """Compiler toolchain with dummy GNU compilers (gcc)."""
    NAME = "testdummy"
    COMPILER_MODULE_NAME = ['GCC']
    SUBTOOLCHAIN = SYSTEM_TOOLCHAIN_NAME
```

I am driving eb with python3 (3.6.9):

```
EB_VERBOSE=1 eb --version
>> Considering '/usr/bin/python3'...
>> '/usr/bin/python3' version: 3.6.9, which matches Python 3 version requirement (>= 3.5)
>> Selected Python command: /usr/bin/python3 (/usr/bin/python3)
>> /usr/bin/python3 -m easybuild.main --version
This is EasyBuild 4.3.0 (framework: 4.3.0, easyblocks: 4.3.0)
```

Current configuration:
```
$ eb --show-config
#
# Current EasyBuild configuration
# (C: command line argument, D: default value, E: environment variable, F: configuration file)
#
buildpath          (F) = /software/build
containerpath      (F) = /software/containers
hide-deps          (F) = Autoconf, Automake, Autotools, Bison, M4, XZ, binutils, expat, flex, gettext, help2man, libpciaccess, libtool, ncurses, numactl, xorg-macros, zlib, cURL, pkg-config, Szip, CMake, Doxygen, libxml2, JasPer, YAXT, ecCodes, X11, fontconfig, freetype, libpng, util-linux, gperf, Java, intltool, XML-Parser, bzip2, libreadline, xproto, makedepend, NASM, libjpeg-turbo, g2lib, g2clib, libiconv, LibTIFF, GMP, pixman, Tcl, PCRE, SQLite, GSL, PROJ, libffi, SWIG, GEOS, libgeotiff, GDAL, GLib, cairo
hooks              (F) = /software/easyconfigs/kup_custom_hooks.py
include-toolchains (F) = /software/custom_toolchains/*.py, /software/custom_toolchains/compiler/*.py, /software/custom_toolchains/mpi/*.py
installpath        (F) = /software
packagepath        (F) = /software/packages
prefix             (F) = /software
repositorypath     (F) = /software/ebfiles_repo
robot-paths        (F) = /software/easyconfigs, /software/software/EasyBuild/4.3.0/easybuild/easyconfigs
sourcepath         (F) = /software/sources
```

Full debug log (DEBUG_EASYBUILD_OPTIONS=1 eb --list-toolchains) : https://gist.github.com/gujans/86a6961e45d31f87ffb8fbcdfeacd137


Discussion:
I can't reproduce this with the v4.3.0 branch using Python 3.6
```
[ocaisa@generoso EasyBuild_Git]$ EB_VERBOSE=1 eb --include-toolchains=testdummy.py --list-toolchains
>> Considering 'python'...
>> No 'python' found in $PATH, skipping...
>> Considering 'python3'...
>> 'python3' version: 3.6.8, which matches Python 3 version requirement (>= 3.5)
>> Selected Python command: python3 (/usr/bin/python3)
>> python3 -m easybuild.main --include-toolchains=testdummy.py --list-toolchains
List of known toolchains (toolchainname: module[,module...]):
	ClangGCC: Clang, GCC
	CrayCCE: PrgEnv-cray
```
Could it have something to do with your hooks?
Sorry for the slow reply. I have tried to reproduce this myself and in my staging environment it works just fine now. Including our custom hooks. Weird.

Since I ended up not using a custom toolchain after all in the productive environment and I am unable to reproduce this I suggest we close this issue until it appears again for someone where it actually causes issues?
I'm also hitting this same crash in the EasyBuild framework tests, but strangely only when testing with Python 3.9...

Here's the full traceback:

```
Traceback (most recent call last):
  File "/home/runner/work/easybuild-framework/easybuild-framework/easybuild/tools/options.py", line 1353, in parse_options
    eb_go = EasyBuildOptions(usage=usage, description=description, prog=\'eb\', envvar_prefix=CONFIG_ENV_VAR_PREFIX,
  File "/home/runner/work/easybuild-framework/easybuild-framework/easybuild/tools/options.py", line 242, in __init__
    super(EasyBuildOptions, self).__init__(*args, **kwargs)
  File "/home/runner/work/easybuild-framework/easybuild-framework/easybuild/base/generaloption.py", line 924, in __init__
    self.postprocess()
  File "/home/runner/work/easybuild-framework/easybuild-framework/easybuild/tools/options.py", line 856, in postprocess
    self._postprocess_list_avail()
  File "/home/runner/work/easybuild-framework/easybuild-framework/easybuild/tools/options.py", line 1099, in _postprocess_list_avail
    msg += list_toolchains(self.options.output_format)
  File "/home/runner/work/easybuild-framework/easybuild-framework/easybuild/tools/docs.py", line 730, in list_toolchains
    tclist = sorted(zip(all_tcs_names, all_tcs))
TypeError: \'<\' not supported between instances of \'type\' and \'type\'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/opt/hostedtoolcache/Python/3.9.0/x64/lib/python3.9/runpy.py", line 197, in _run_module_as_main
    return _run_code(code, main_globals, None,
  File "/opt/hostedtoolcache/Python/3.9.0/x64/lib/python3.9/runpy.py", line 87, in _run_code
    exec(code, run_globals)
  File "/home/runner/work/easybuild-framework/easybuild-framework/easybuild/main.py", line 533, in <module>
    main()
  File "/home/runner/work/easybuild-framework/easybuild-framework/easybuild/main.py", line 204, in main
    eb_go, cfg_settings = set_up_configuration(args=args, logfile=logfile, testing=testing)
  File "/home/runner/work/easybuild-framework/easybuild-framework/easybuild/tools/options.py", line 1394, in set_up_configuration
    eb_go = parse_options(args=args)
  File "/home/runner/work/easybuild-framework/easybuild-framework/easybuild/tools/options.py", line 1357, in parse_options
    raise EasyBuildError("Failed to parse configuration options: %s" % err)
easybuild.tools.build_log.EasyBuildError: "Failed to parse configuration options: \'<\' not supported between instances of \'type\' and \'type\'"
```

Resolving patch:
diff --git a/easybuild/tools/docs.py b/easybuild/tools/docs.py
--- a/easybuild/tools/docs.py
+++ b/easybuild/tools/docs.py
@@ -726,16 +726,16 @@ def list_software_txt(software, detailed=False):
 def list_toolchains(output_format=FORMAT_TXT):
     """Show list of known toolchains."""
     _, all_tcs = search_toolchain('')
-    all_tcs_names = [x.NAME for x in all_tcs]
-    tclist = sorted(zip(all_tcs_names, all_tcs))
 
-    tcs = dict()
-    for (tcname, tcc) in tclist:
+    # filter deprecated 'dummy' toolchain
+    all_tcs = [x for x in all_tcs if x.NAME != DUMMY_TOOLCHAIN_NAME]
+    all_tcs_names = [x.NAME for x in all_tcs]
 
-        # filter deprecated 'dummy' toolchain
-        if tcname == DUMMY_TOOLCHAIN_NAME:
-            continue
+    # start with dict that maps toolchain name to corresponding subclass of Toolchain
+    tcs = dict(zip(all_tcs_names, all_tcs))
 
+    for tcname in sorted(tcs):
+        tcc = tcs[tcname]
         tc = tcc(version='1.2.3')  # version doesn't matter here, but something needs to be there
         tcs[tcname] = tc.definition()
 


Test patch: