Repository: spack/spack

Issue:
"spack load" broken with error "bash: /Modules/bin/modulecmd: No such file or directory"
Apparently, the current spack in the repo is broken on Ubuntu 18.04

Following the turorial fails:

```
git clone https://github.com/spack/spack.git
. spack/share/spack/setup-env.sh
spack install hdf5
```
`git clone https://github.com/spack/spack.git
. spack/share/spack/setup-env.sh
spack install hdf5`

Besides reporting the issue, I have the following questions:

1. What can I do to recover my already compiled modules? I really hate to wait another day to have them compiled again, once downgrade to a working version of spack.
2. Do you plan to start publishing a "stable" version even at the cost of a little outdated module?


Discussion:
Version `20a89ca326712058f3f22621eed725c0f510bee3` works well. 
@alalazo 
Correction: `20a89ca326712058f3f22621eed725c0f510bee3` actually doesn't work. It may be something broken with my 18.04 system. Could you tell me, what it may be? Obviously, I don't have "Modules" in the root directory, but it shouldn't be a problem ;-). 

Running spack with `--debug` doesn't tell anything more:
```
spack --debug load tcl
==> READ LOCK: /home/Docs/Adam/linux/tmp/spack/opt/spack/.spack-db/lock[0:0] [Acquiring]
==> READ LOCK: /home/Docs/Adam/linux/tmp/spack/opt/spack/.spack-db/lock[0:0] [Acquired]
==> Reading config file /home/Docs/Adam/linux/tmp/spack/etc/spack/defaults/repos.yaml
==> READ LOCK: /home/Docs/Adam/linux/tmp/spack/opt/spack/.spack-db/lock[0:0] [Released]
bash: /Modules/bin/modulecmd: No such file or directory
```
Not sure what changed. In the meantime, you could test out #8570.
@adamjstewart I tried #8570, but nothing changed:
```
$ cd spack
$ git checkout ac0c14688ebce72cafb6ce0dd23fa00f5582635a
$ . /home/Docs/Adam/linux/tmp/spack/share/spack/setup-env.sh #loads spack
$ spack --debug load tcl 
==> READ LOCK: /home/Docs/Adam/linux/tmp/spack/opt/spack/.spack-db/lock[0:0] [Acquiring]
==> READ LOCK: /home/Docs/Adam/linux/tmp/spack/opt/spack/.spack-db/lock[0:0] [Acquired]
==> Reading config file /home/Docs/Adam/linux/tmp/spack/etc/spack/defaults/repos.yaml
==> READ LOCK: /home/Docs/Adam/linux/tmp/spack/opt/spack/.spack-db/lock[0:0] [Released]
bash: /Modules/bin/modulecmd: No such file or directory
```

@becker33 
Installing `environment-modules` (an Ubuntu package that contains `modulecmd`) does not help either:

on #8570 branch:

```
$ sudo apt install environment-modules
$ . /home/Docs/Adam/linux/tmp/spack/share/spack/setup-env.sh 
$ spack --debug load tcl 
==> READ LOCK: /home/Docs/Adam/linux/tmp/spack/opt/spack/.spack-db/lock[0:0] [Acquiring]
==> READ LOCK: /home/Docs/Adam/linux/tmp/spack/opt/spack/.spack-db/lock[0:0] [Acquired]
==> Reading config file /home/Docs/Adam/linux/tmp/spack/etc/spack/defaults/repos.yaml
==> READ LOCK: /home/Docs/Adam/linux/tmp/spack/opt/spack/.spack-db/lock[0:0] [Released]
bash: /Modules/bin/modulecmd: No such file or directory
```

The same error on the `develop`:

```
$ . /home/Docs/Adam/linux/tmp/spack/share/spack/setup-env.sh 
$ spack --debug load tcl 
==> READ LOCK: /home/Docs/Adam/linux/tmp/spack/opt/spack/.spack-db/lock[0:0] [Acquiring]
==> READ LOCK: /home/Docs/Adam/linux/tmp/spack/opt/spack/.spack-db/lock[0:0] [Acquired]
==> READ LOCK: /home/Docs/Adam/linux/tmp/spack/opt/spack/.spack-db/lock[0:0] [Released]
bash: /Modules/bin/modulecmd: No such file or directory
```

actually I've got the same problem with my fresh installed CentOS 7, and I don't think problem is much related to your ubuntu 18.04 :)  @adamryczkowski 

`spack load` worked fine two days ago when I tried, so I `git reset --hard da5663a` and everything goes well again.
Can you see if it's related to @tgamblin's #8101? That's the only module-related thing that's been merged in the last 2 days. Just check out the commit before and after and see if that is indeed what broke.
After restarting the system, the error message changed from `bash: /Modules/bin/modulecmd: No such file or directory` to `module: Command not found`. Hmm...
Commit `da5663a` seems to work after the reset. At last I can return to my development.

@adamjstewart I will check commits #8101 after I am done with the postponed work.
@adamjstewart @tgamblin I could reproduce the issue by cloning Spack and:
```console
$ spack bootstrap
$ . share/spack/setup-env.sh 
$ module av
bash: /Modules/bin/modulecmd: No such file or directory
```
Using `git bisect`:
```console
$ git bisect good
06418a3dcd5dda0c45151ceec08aa91d8e3fa651 is the first bad commit
commit 06418a3dcd5dda0c45151ceec08aa91d8e3fa651
Author: Todd Gamblin <tgamblin@llnl.gov>
Date:   Mon Jul 16 15:43:44 2018 -0700

    add `spack --print-shell-vars` to speed up `setup-env.[c]sh` (#8101)
    
    - The setup-env.sh script currently makes two calls to spack, but it
      should only need to make one.
    
    - Add a fast-path shell setup routine in `main.py` to allow the shell
      setup to happen in a single, fast call that doesn't load more than it
      needs to.
    
    - This simplifies setup code, as it has to eval what Spack prints
    
    - TODO: consider eventually making the whole setup script the output of a
      spack command

:040000 040000 5b4711f3d61e42cc87efae605a78c07c1c9ef9ec 60ff517d3f377bd6b0a42ea72ef24c3a37fdb7e8 M	lib
:040000 040000 59ece4f0723278a8b624b83dd3ccd067d4eb31d3 2d70f6d31e8616a2f23594982d1b49ef71ed2874 M	share
```
~Further info: #8101 hooking to cli seems a bit broken:~
```
$ spack --print-shell-vars sh,modules
==> Error: Unknown command: sh,modules
```

EDIT: Maybe it's better for me to check-out again develop after bisect :smile: 
perhaps this problem is caused by **_sp_module_prefix** in `share/spack/setup-env.sh: 229` and `231`, which maybe typos. @adamjstewart 

when I modified file `share/spack/setup-env.sh` and printed `_sp_module_prefix`, it showed nothing.

then I tried `spack --print-shell-vars sh,modules`, and the output showed that there was no `_sp_module_prefix`:

```
_sp_sys_type='linux-centos7-x86_64'
_sp_tcl_root='/home/ABC/spack/share/spack/modules'
_sp_lmod_root='/home/ABC/spack/share/spack/lmod'
_sp_dotkit_root='/home/ABC/spack/share/spack/dotkit'
module_prefix='/home/ABC/spack/opt/spack/linux-centos7-x86_64/gcc-4.8.5/environment-modules-3.2.10-zdnuljvtjsvdx5nrnfrjg4oziu2izut4'
```

so i did followings based on commit **25062d**:
1) changed  `_sp_module_prefix` of lines `share/spack/setup-env.sh: 229` and `231` into **module_prefix**;
2) opened a new terminal, and tried `. spack/share/spack/setup-env.sh` and `spack load`, it worked again.



@xpsair I'd rather change the output of `spack --print-shell-vars sh,modules` to return `_sp_module_prefix`. Would you like to submit a PR to fix this?

Resolving patch:
diff --git a/lib/spack/spack/main.py b/lib/spack/spack/main.py
--- a/lib/spack/spack/main.py
+++ b/lib/spack/spack/main.py
@@ -589,9 +589,9 @@ def shell_set(var, value):
     if 'modules' in info:
         specs = spack.store.db.query('environment-modules')
         if specs:
-            shell_set('module_prefix', specs[-1].prefix)
+            shell_set('_sp_module_prefix', specs[-1].prefix)
         else:
-            shell_set('module_prefix', 'not_installed')
+            shell_set('_sp_module_prefix', 'not_installed')
 
 
 def main(argv=None):


Test patch:
diff --git a/lib/spack/spack/test/cmd/print_shell_vars.py b/lib/spack/spack/test/cmd/print_shell_vars.py
--- a/lib/spack/spack/test/cmd/print_shell_vars.py
+++ b/lib/spack/spack/test/cmd/print_shell_vars.py
@@ -32,7 +32,7 @@ def test_print_shell_vars_sh(capsys):
     assert "_sp_sys_type=" in out
     assert "_sp_tcl_root=" in out
     assert "_sp_lmod_root=" in out
-    assert "module_prefix" not in out
+    assert "_sp_module_prefix" not in out
 
 
 def test_print_shell_vars_csh(capsys):
@@ -42,7 +42,7 @@ def test_print_shell_vars_csh(capsys):
     assert "set _sp_sys_type = " in out
     assert "set _sp_tcl_root = " in out
     assert "set _sp_lmod_root = " in out
-    assert "set module_prefix = " not in out
+    assert "set _sp_module_prefix = " not in out
 
 
 def test_print_shell_vars_sh_modules(capsys):
@@ -52,7 +52,7 @@ def test_print_shell_vars_sh_modules(capsys):
     assert "_sp_sys_type=" in out
     assert "_sp_tcl_root=" in out
     assert "_sp_lmod_root=" in out
-    assert "module_prefix=" in out
+    assert "_sp_module_prefix=" in out
 
 
 def test_print_shell_vars_csh_modules(capsys):
@@ -62,4 +62,4 @@ def test_print_shell_vars_csh_modules(capsys):
     assert "set _sp_sys_type = " in out
     assert "set _sp_tcl_root = " in out
     assert "set _sp_lmod_root = " in out
-    assert "set module_prefix = " in out
+    assert "set _sp_module_prefix = " in out