Repository: spack/spack

Issue:
test_get_module_cmd_fails failing in develop
@tgamblin @alalazo @becker33 I assume this is related to the recent merge of #3250.

If you look at the unit tests for #4568 you'll see a lot of unrelated unit tests failures in Travis. This PR simply adds a new package, so the unit tests should not be failing. Based on the error messages, I'm guessing there's a Python 3 byte incompatible thing we're doing when parsing modulecmd output.


Discussion:
@adamjstewart You're right: we have a broken develop. I think we should start to take the habit of rebasing on the latest develop tip before merging any modification to the core. There's also a way to require this for every PR, but maybe this is too much of a burden for packages.
> There's also a way to require this for every PR

I ran into a project that did this and it was an absolute nightmare. So downvote from me. But yeah, big PRs that affect internal things would be good candidates for a rebasing once in a while.
Also, codecov plummeted to 55%? That's weird.
@adamjstewart Not weird at all: we upload the coverage files *after success*
Based on the error message, I'm guessing this bug was caused by the combination of #4230 and #3250:
```
$ spack test -s -k module_parsing
==================================== test session starts =====================================
platform linux2 -- Python 2.7.13, pytest-3.0.5, py-1.4.32, pluggy-0.4.0
rootdir: /home/ajstewart/spack, inifile: pytest.ini
collected 753 items 

lib/spack/spack/test/module_parsing.py .....==> Warning: Could not detect module function from bash. Trying to detect modulecmd from `which`
F==> Warning: Could not detect module function from bash. Trying to detect modulecmd from `which`
.
================================== short test summary info ===================================
FAIL lib/spack/spack/test/module_parsing.py::test_get_module_cmd_fails

================================= slowest 20 test durations ==================================
0.05s call     lib/spack/spack/test/module_parsing.py::test_get_path_from_module
0.03s call     lib/spack/spack/test/module_parsing.py::test_get_module_cmd_from_bash_using_modules
0.02s call     lib/spack/spack/test/module_parsing.py::test_get_module_cmd_from_bash_ticks
0.02s call     lib/spack/spack/test/module_parsing.py::test_get_module_cmd_from_bash_parens
0.02s call     lib/spack/spack/test/module_parsing.py::test_get_module_cmd_from_which
0.01s call     lib/spack/spack/test/module_parsing.py::test_get_module_cmd_fails
0.00s setup    lib/spack/spack/test/module_parsing.py::test_get_module_cmd_from_bash_ticks
0.00s setup    lib/spack/spack/test/module_parsing.py::test_get_module_cmd_from_which
0.00s setup    lib/spack/spack/test/module_parsing.py::test_get_module_cmd_fails
0.00s setup    lib/spack/spack/test/module_parsing.py::test_get_module_cmd_from_bash_parens
0.00s setup    lib/spack/spack/test/module_parsing.py::test_get_module_cmd_from_bash_using_modules
0.00s setup    lib/spack/spack/test/module_parsing.py::test_get_argument_from_module_line
0.00s setup    lib/spack/spack/test/module_parsing.py::test_get_path_from_module
0.00s teardown lib/spack/spack/test/module_parsing.py::test_get_module_cmd_from_bash_parens
0.00s teardown lib/spack/spack/test/module_parsing.py::test_get_module_cmd_from_bash_ticks
0.00s call     lib/spack/spack/test/module_parsing.py::test_get_argument_from_module_line
0.00s teardown lib/spack/spack/test/module_parsing.py::test_get_module_cmd_from_which
0.00s teardown lib/spack/spack/test/module_parsing.py::test_get_path_from_module
0.00s teardown lib/spack/spack/test/module_parsing.py::test_get_module_cmd_from_bash_using_modules
0.00s teardown lib/spack/spack/test/module_parsing.py::test_get_argument_from_module_line
========================================== FAILURES ==========================================
_________________________________ test_get_module_cmd_fails __________________________________

save_env = None

    def test_get_module_cmd_fails(save_env):
        os.environ.pop('BASH_FUNC_module()')
        os.environ.pop('PATH')
        with pytest.raises(ModuleError):
>           module_cmd = get_module_cmd(b'--norc')

lib/spack/spack/test/module_parsing.py:127: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
lib/spack/spack/util/module_cmd.py:44: in get_module_cmd
    return get_module_cmd_from_which()
lib/spack/spack/util/module_cmd.py:53: in get_module_cmd_from_which
    module_cmd = which('modulecmd')
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

args = ('modulecmd',), kwargs = {}, path = None, required = False, name = 'modulecmd'

    def which(*args, **kwargs):
        """Finds an executable in the path like command-line which.
    
        If given multiple executables, returns the first one that is found.
        If no executables are found, returns None.
    
        Parameters:
            *args (str): One or more executables to search for
    
        Keyword Arguments:
            path (:func:`list` or str): The path to search. Defaults to ``PATH``
            required (bool): If set to True, raise an error if executable not found
    
        Returns:
            Executable: The first executable that is found in the path
        """
        path = kwargs.get('path', os.environ.get('PATH'))
        required = kwargs.get('required', False)
    
        if isinstance(path, string_types):
            path = path.split(os.pathsep)
    
        for name in args:
>           for directory in path:
E           TypeError: 'NoneType' object is not iterable

lib/spack/spack/util/executable.py:249: TypeError
==================================== 746 tests deselected ====================================
===================== 1 failed, 6 passed, 746 deselected in 1.02 seconds =====================
```
I agree with @adamjstewart -- we'll just churn travis if we always rebase.  We had a discussion about this and kind of decided that we were ok with the occasional break.

We should revert whichever merge broke `develop`, then rebase *that* and fix it.
See my comment on #4578

Summary: The cause is no default case in `which` for an empty path variable. I will submit a PR once the tests complete locally

Resolving patch:
diff --git a/lib/spack/spack/util/executable.py b/lib/spack/spack/util/executable.py
--- a/lib/spack/spack/util/executable.py
+++ b/lib/spack/spack/util/executable.py
@@ -239,7 +239,7 @@ def which(*args, **kwargs):
     Returns:
         Executable: The first executable that is found in the path
     """
-    path = kwargs.get('path', os.environ.get('PATH'))
+    path = kwargs.get('path', os.environ.get('PATH', ''))
     required = kwargs.get('required', False)
 
     if isinstance(path, string_types):


Test patch: