Repository: python/mypy

Issue:
problem finding data_dir on Windows when run using "python -m mypy"
I ran into issues with mypy on a simple example. Running Windows 8.1, Python 3.5 using Anaconda. Installed mypy using `pip install git+https://github.com/python/mypy.git`
Saved the following as `a.py`:

``` python
import typing

def f():
    1 + 'x'  # No static type error (dynamically typed)

def g() -> None:
    1 + 'x'  # Type check error (statically typed)
```

Ran the following in the command prompt: `python -m mypy a.py` and got the following:

```
a.py:1: error: Could not find builtins
a.py:1: error: No library stub file for standard library module 'typing'
a.py:1: note: (Stub files are from https://github.com/python/typeshed)
```

I found that the files were installed in the following locations:
- The `.py` files are in `C:/Users/MyUserName/Anaconda3/Lib/site-packages/mypy`
- The `typeshed` and `xml` directories are in `C:/Users/MyUserName/Anaconda3/Lib/mypy`
- The mypy script is `C:/Users/MyUserName/Anaconda3/Scripts/mypy` (Running mypy in the command prompt doesn't work in Windows, which is why I had to run it as a module.)

I ran pdb and found that when I call mypy as a module:
1. `__init__.py` calls mypy.main.main(None)
2. `build.build` is called with `bin_dir=None`
3. `build.default_data_dir` is called with `bin_dir=None`
4. `build.default_data_dir` returns `C:/Users/MyUserName/Anaconda3/Lib/site-packages` when it should return `C:/Users/MyUserName/Anaconda3/Lib/mypy`

In `build.default_data_dir`, for the case of `not bin_dir` I changed the line `lib = os.path.dirname(os.path.dirname(parent))` to `lib = os.path.dirname(parent)` and now I can run mypy as a module. Not sure if it's the best solution but it works.

Possibly related issue: https://github.com/python/mypy/issues/653. Later realized that I can run the mypy script within git bash if I change `#!/usr/bin/env python3` to `#!/usr/bin/env python`, and then I don't have to worry about the module stuff. But Windows users installing from pip may not have a bash shell.



Discussion:
On Windows using CPython 3.5.1 in a venv I'm also running into the `ni\__init__.py:1: error: Could not find builtins` message. The package I want checked is in the same directory I'm running from using the command, .e.g. `.\venv\Scripts\python.exe -m mypy ni`.

Maybe a fix similar to #1253 is needed, or a generalization thereof. I can't test on Windows but I'm happy to take PRs from people who are in a position to test them.

Starting to dig into this for CPython 3.5.1 on Windows 10 in a venv, it looks like https://github.com/python/mypy/blob/d4a555bfa7016432940f38e2029d03e6bae81f18/mypy/build.py#L767 is where the error message is being specified, so that suggests the `builtins` module can't be typed. OK, so that's a stdlib module, so I guess typeshed should be handling this? Anyway, when I print out https://github.com/python/mypy/blob/d4a555bfa7016432940f38e2029d03e6bae81f18/mypy/build.py#L1028 for `builtins` I get `('builtins', ('C:\\Users\\brcan\\Documents\\Repositories\\knights-who-say-ni', '.', '.', '.', '.', '.
', '.', '.', '.', '.', '.', '.'))` which obviously contains no path to typeshed as installed in the venv.

Now looking at my venv, the location of `typeshed` in the venv is `.\venv\Lib\mypy` where there is a `typeshed` and `xml` directory (this is also where some `tcl8.6` in `.\venv\Lib` so this seems to be where `pip` and `setuptools` puts your data files). When I looked at `setup.py` I see you specified the `typeshed` files as `data_files` in `setup()`, but then I also noticed you don't use `pkg_resources` or `setuptools` in your code to calculate the path to `typeshed`. So my suspicion is you're making some assumptions somewhere about where `typeshed` files are being installed that are not followed on Windows because you're hand-calculating the paths to data files instead of using `pkg_resources` and/or `setuptools` to do it for you.

Correct! The PR that I linked to actually shows the code where this determination is made. Sorry, I should have been more explicit. Here's the logic: https://github.com/python/mypy/blob/d4a555bfa7016432940f38e2029d03e6bae81f18/mypy/build.py#L215


Resolving patch:
diff --git a/mypy/build.py b/mypy/build.py
--- a/mypy/build.py
+++ b/mypy/build.py
@@ -213,17 +213,26 @@ def build(sources: List[BuildSource],
 
 
 def default_data_dir(bin_dir: str) -> str:
-    # TODO fix this logic
+    """Returns directory containing typeshed directory
+
+    Args:
+      bin_dir: directory containing the mypy script
+    """
     if not bin_dir:
         mypy_package = os.path.dirname(__file__)
         parent = os.path.dirname(mypy_package)
-        if os.path.basename(parent) == 'site-packages':
-            # Installed in site-packages, but invoked with python3 -m mypy;
-            # __file__ is .../blah/lib/python3.N/site-packages/mypy/__init__.py;
+        if (os.path.basename(parent) == 'site-packages' or
+                os.path.basename(parent) == 'dist-packages'):
+            # Installed in site-packages or dist-packages, but invoked with python3 -m mypy;
+            # __file__ is .../blah/lib/python3.N/site-packages/mypy/build.py
+            # or .../blah/lib/python3.N/dist-packages/mypy/build.py (Debian)
+            # or .../blah/lib/site-packages/mypy/build.py (Windows)
             # blah may be a virtualenv or /usr/local.  We want .../blah/lib/mypy.
-            lib = os.path.dirname(os.path.dirname(parent))
-            if os.path.basename(lib) == 'lib':
-                return os.path.join(lib, 'mypy')
+            lib = parent
+            for i in range(2):
+                lib = os.path.dirname(lib)
+                if os.path.basename(lib) == 'lib':
+                    return os.path.join(lib, 'mypy')
         subdir = os.path.join(parent, 'lib', 'mypy')
         if os.path.isdir(subdir):
             # If installed via buildout, the __file__ is
@@ -240,7 +249,7 @@ def default_data_dir(bin_dir: str) -> str:
         return os.path.join(dir, 'Lib', 'mypy')
     elif base == 'scripts':
         # Assume that we have a repo check out or unpacked source tarball.
-        return os.path.dirname(bin_dir)
+        return dir
     elif base == 'bin':
         # Installed to somewhere (can be under /usr/local or anywhere).
         return os.path.join(dir, 'lib', 'mypy')


Test patch: