Repository: python/mypy

Issue:
RuntimeError: Broken installation: can't determine base dir
When installing via portage (I'm working on packaging mypy for portage), I get the following when trying to run mypy:

```
Traceback (most recent call last):
  File "/usr/lib/python-exec/python3.3/mypy", line 169, in <module>
    main()
  File "/usr/lib/python-exec/python3.3/mypy", line 34, in main
    type_check_only(path, module, bin_dir, options)
  File "/usr/lib/python-exec/python3.3/mypy", line 78, in type_check_only
    python_path=options.python_path)
  File "/usr/lib64/python3.3/site-packages/mypy/build.py", line 122, in build
    data_dir = default_data_dir(bin_dir)
  File "/usr/lib64/python3.3/site-packages/mypy/build.py", line 191, in default_data_dir
    raise RuntimeError("Broken installation: can't determine base dir")
RuntimeError: Broken installation: can't determine base dir
```

I'm not sure what directory it's starting with but I can tell it's not finding what it's looking for.  Any guidance would be appreciated.



Discussion:
Can you give me the directories where different files from mypy live in your setup? Currently mypy is fairly rigid about where different files should live, but I'm happy to update this to support new configurations.

In particular, mypy needs to find the stub files.

Hey @JukkaL ,

Looks like those are in /usr/lib64.  Here's a paste of every file installed:

https://bpaste.net/show/52a8a881523e

Thanks for the info! I'll look into this, hopefully later this week. It should be pretty easy to fix.

@JukkaL Thanks!  Let me know if you need anything else.

Looks like another Gentoo user found a solution in this bug: https://bugs.gentoo.org/show_bug.cgi?id=551272

I'm going to apply the patch to our ebuild and can submit it as a pull request if it's acceptable.  Otherwise, let me know what a better solution would be.

Apologies for not making progress on this. The fix in the linked issue doesn't work on Python 3.2, which we still support.

BTW, the paste link above apparently has expired.

@JukkaL,  Thanks for looking into this again.  Here's a new paste link and let me know if I can provide anything else.

https://bpaste.net/show/31ff1a7d0968

I'm facing this error too. It may be caused by below commit.

2c009ca991bf5d1b42741c0d1a2fbf6bb6077cd4

The change from `scripts` to `mypy` is not needed. Because `mypy` script exists under the `Scripts` folder in Windows (below is the point of change).

https://github.com/JukkaL/mypy/commit/2c009ca991bf5d1b42741c0d1a2fbf6bb6077cd4#diff-564bf8da9227d198a635daea8a663e2bR173

mypy works fine when I reverted this change.

@icoxfog417: Can you give me more details about your configuration (OS, Python version, are you using virtualenv, where are Python and mypy installed, which package did you use for Python)?

@JukkaL I'm sorry fot the lack of information.
- OS: Windows 8.1
- Python version: 3.5 and using [Conda](http://conda.pydata.org/)
- The location that mypy command installed: (virtual environment root)/Scripts/mypy
- mypy version: latest (from Github master branch (08dfca35994cd40f41b01f32bbff6368a5d51999))
  
  I tried it by virtualenv/Python 3.4, but the result was same.

And also, Windows doesn't have `python3`. So mypy script fails when using bash program (like msys2, cygwin).

```
#!/usr/bin/env python3
"""Mypy type checker command line tool."""

from mypy.main import main

main(__file__)
```

I'm still seeing this issue on Gentoo now that the aforementioned patch no longer applies to master.  I can provide more information but I imagine it's the same as before.  The current stack trace (slightly different):

```
% mypy --use-python-path -m MODULE
Traceback (most recent call last):
  File "/usr/lib/python-exec/python3.4/mypy", line 6, in <module>
    main(__file__)
  File "/usr/lib64/python3.4/site-packages/mypy/main.py", line 46, in main
    git.verify_git_integrity_or_abort(build.default_data_dir(bin_dir))
  File "/usr/lib64/python3.4/site-packages/mypy/build.py", line 224, in default_data_dir
    raise RuntimeError("Broken installation: can't determine base dir")
RuntimeError: Broken installation: can't determine base dir
```

@alunduil Hmm.. some more information would be useful. Can you tell where the stub files are? For example, run this command and give me the output:

```
$ find /usr -name builtins.pyi
```

@icoxfog417 Sorry, I dropped the ball on your issue. I'll try to find time to do some testing on Windows later this week or the next week. If possible, it would be helpful if you can tell me where the builtins.pyi file lives in your installation.

@JukkaL 

Here's the results of that find command:

```
alunduil@elijah ~ % find /usr -name builtins.pyi
/usr/lib64/mypy/typeshed/builtins/3/builtins.pyi
/usr/lib64/mypy/typeshed/builtins/2.7/builtins.pyi
```

Let me know if any other information will help.  I'm sure I didn't provide everything you need to troubleshoot the issue well.

My environment is as below. `mypy` works with some fixes.
- Windows 7/32bit (Windows8.1 also)
- Python 3.5
- mypy version: latest (`pip install git+https://github.com/JukkaL/mypy.git`)
- running on Git Shell (msys2)

There are some points.
- I use latest mypy (`master` branch) because mypy on PyPI doesn't support Python 3.5 (PyPI version causes `ImportError: cannot import name 'Undefined'` due to the removing of `Undefined` type (please refer #639)).
- I have to modify [`mypy` script file's header](https://github.com/JukkaL/mypy/blob/master/scripts/mypy#L1) from `#!/usr/bin/env python3` to `#!/usr/bin/env python` because that Windows doesn't have `python3` command (If you use command prompt, it doesn't matter).

And the `builtins.pyi` is located in `(my virtualenv folder)\Lib\mypy\typeshed\builtins\3`.

Then `mypy` works.

```
$ mypy exmaple.py
exmaple.py:5: error: Argument 1 to "sum" has incompatible type "float"; expected "int"
```

`example.py` is like below.

```
def sum(x: int, y:int) -> int:
    return x + y

if __name__ == "__main__":
    print(sum(0.1, 2))  # causes type check error
```

@JukkaL, is there an easy fix for this that I can submit a pull request for?  Why are paths hard coded rather than determined relative to a module or searched for in the lib search path?

The (somewhat) hard-coded paths are because installers put "data" files (in mypy's case that's the stub files) in different locations depending on platform and installer policies, and these locations aren't directly relative to where the code (the mypy package) is installed.

I guess if we could arrange to always install the stubs inside the "mypy" module that would make finding them easier (using `pkgutil.get_data()`), but this would require substantial changes to mypy's search for stub files. The current approach to finding stub files requires them to be in the filesystem, and that's why they are treated as "data".

That makes sense.  Would you suggest adding the appropriate path for this issue or to look into revamping the stub file search code?  I'm guessing the former since it's a simpler patch but that latter seems more robust.

Yeah, I would just hard-code another path in the existing code (it just
probes various locations until it finds one). It should be a simple PR.
Thanks!

Found the culprit.  It appears that following the symlinks that the mypy script is behind leads to `/usr/lib/python-exec/python-exec?` (where ? is the version, 2 or 3).  This is due to Gentoo's use of dev-python/python-exec to handle multiple python installations and dynamic changes to the default implementation.

This base directory wasn't accounted for in the verify function in the git module.  I'm testing a patch now and will submit a pull request if it's successful.

Great news, looking forward to the PR!


Resolving patch:
diff --git a/mypy/build.py b/mypy/build.py
--- a/mypy/build.py
+++ b/mypy/build.py
@@ -285,6 +285,9 @@ def default_data_dir(bin_dir: str) -> str:
     elif base == 'python3':
         # Assume we installed python3 with brew on os x
         return os.path.join(os.path.dirname(dir), 'lib', 'mypy')
+    elif dir.endswith('python-exec'):
+        # Gentoo uses a python wrapper in /usr/lib to which mypy is a symlink.
+        return os.path.join(os.path.dirname(dir), 'mypy')
     else:
         # Don't know where to find the data files!
         raise RuntimeError("Broken installation: can't determine base dir")


Test patch: