Repository: easybuilders/easybuild-easyblocks

Issue:
EB_PYTHON startup behaviour
The documentation at https://easybuild.readthedocs.io/en/latest/Python-2-3-compatibility.html?#controlling-which-python-command-easybuild-will-use-via-eb-python suggests that one can control which version of Python EasyBuild will use by setting EB_PYTHON.

Since https://github.com/easybuilders/easybuild-easyblocks/commit/40b76bef5a1f4f149e0c13cce913c051b54da5a3 running `module load EasyBuild` will now overwrite any existing EB_PYTHON environment variable.

While it is still possible to configure EB_PYTHON after running `module load EasyBuild`, it is no longer possible to configure EB_PYTHON first (e.g. in one's shell startup), since it is overwritten when EasyBuild loads.

If this is the desired behaviour then it should be documented. If this is not the desired behaviour, then it is a bug.


Discussion:
The `EasyBuild` module sets `$EB_PYTHON` because the Python version that should be used for running EasyBuild is known during the installation. It's a mechanism to prevent that the EasyBuild installation breaks if it was installed with `python3` for example, and then a `python` command gets added to the system afterwards.

Setting `$EB_PYTHON` manually is mostly interesting for EasyBuild installations that are done manually via `pip install` or by cloning the repositories and updated `$PYTHONPATH`, I guess.

I'm happy to try and clarify this in the documentation. Do you have any suggestions on the wording we could use for  that?
Hi again,

The current documentation strongly suggests that EB_PYTHON is used for run-time configuration of which python binary to use.

- The example given in the documentation shows setting EB_PYTHON after EasyBuild has already been installed.
- The documentation states that there is normally no need to set EB_PYTHON because by default it will try `python`, `python3`, `python2`. Using EB_PYTHON is not, according to the documentation, the default behaviour - rather, it is used to adjust the default behaviour.

In short, the current documentation doesn't suggest at all that EB_PYTHON is an install-time configuration - it suggests the opposite.

Furthermore, if it were true EB_PYTHON needs to be set once at install-time and can't change after that, it would be surprising that it took until version 4.1.2 for the developers to become aware of that.

Overall, this feels like a change (and potentially a breaking change) to the previously-intended use for EB_PYTHON.

In any case, what is the reason for wanting to keep the version of Python fixed to what was used at install time? Will EasyBuild break if run with a different version of Python than was used at install time? If the reason is just to prevent surprises to the user in case of path changes, then I would think most EasyBuild users are sufficiently expert not to find that problematic.

Runtime configuration of which python to use is a useful feature since e.g. other packages that work with EasyBuild such as GC3Pie may require particular versions of python and this might change over time.

Many thanks,

Geoff








Ok, let's try to come to an understanding on this so we can move towards closing this issue. To get things straight, `EB_PYTHON` is indeed used for runtime configuration of EasyBuild, i.e., if I run
```
EB_PYTHON=foo eb ...
```
eb will try to use `foo` for Python when running. What you asking for is something different, you are asking that if you have set `EB_PYTHON` then loading the EasyBuild module should not override that setting. As far as I can tell, there is no explicit way to allow for this behaviour in a module file (unless you use lua/tcl features as opposed to module tool features).

The other option is to have an additional environment variable that takes lower precedence than `EB_PYTHON` and use that in the EasyBuild module instead, something like `EB_INSTALLPYTHON`. That would not require changes in the documentation but will require PRs in the framework (to respect the variable in the `eb` script) and the easyblock repo (to set the variable instead of `EB_PYTHON`) 
Thank you, @ocaisa. I agree with the essentials of your proposal.

Resolving patch:
diff --git a/easybuild/easyblocks/e/easybuildmeta.py b/easybuild/easyblocks/e/easybuildmeta.py
--- a/easybuild/easyblocks/e/easybuildmeta.py
+++ b/easybuild/easyblocks/e/easybuildmeta.py
@@ -219,10 +219,11 @@ def sanity_check_step(self):
 
     def make_module_extra(self):
         """
-        Set $EB_PYTHON to ensure that this EasyBuild installation uses the same Python executable it was installed with.
+        Set $EB_INSTALLPYTHON to ensure that this EasyBuild installation uses the same Python executable it was
+        installed with (which can still be overridden by the user with $EB_PYTHON).
         """
         txt = super(EB_EasyBuildMeta, self).make_module_extra()
-        txt += self.module_generator.set_environment('EB_PYTHON', self.python_cmd)
+        txt += self.module_generator.set_environment('EB_INSTALLPYTHON', self.python_cmd)
         return txt
 
     def make_module_step(self, fake=False):


Test patch: