Repository: python/mypy

Issue:
Cannot get type hints from python 2 packages
In mypy 0.620, there appears to be inconsistent type checking behaviour depending on how you specify the python version. This exists for handling `-2` and `--python-version 2` arguments and is explained well in [issue #5576](https://github.com/python/mypy/issues/5576#issuecomment-418872062) and addressed by pull request #5619. However, this inconsistency also occurs with specifying the python version in the a mypy.ini configuration file.

```
$ mypy --python-version 2.7 pkgb
pkgb/__main__.py:7: error: Argument 1 to "this_is_a_typed_function" has incompatible type "int"; expected "unicode"
```
That above is expected, but when specifying the version in a config file.
```
$ echo -e "[mypy]\npython_version: 2.7" | mypy --config-file /dev/stdin pkgb
pkgb/__main__.py:1: error: Cannot find module named 'pkga'
```

For good measure, I included a python 2 print statement in the test package to ensure it is actually is running under python 2 mode ... [The example is included here](https://github.com/python/mypy/files/2383934/mypy5576.tar.gz). Extract the archive; in a Python 2 virtualenv, install the package in the `a` directory with `pip -e a`; run mypy on `pkgb`.

---

The behaviour seems to come from looking for typing information in different site-packages between the two cases. With `--python-version 2.7`, a Python 2 interpreter's site-packages are used. With the configuration file, the current interpreter's site-packages are used.

The pull request #5578 was an attempt to resolve this inconsistency by using the Python 2 site-packages in both configurations, but it seems like this broke some things for some folks.
Cannot get type hints from python 2 packages
In mypy 0.620, there appears to be inconsistent type checking behaviour depending on how you specify the python version. This exists for handling `-2` and `--python-version 2` arguments and is explained well in [issue #5576](https://github.com/python/mypy/issues/5576#issuecomment-418872062) and addressed by pull request #5619. However, this inconsistency also occurs with specifying the python version in the a mypy.ini configuration file.

```
$ mypy --python-version 2.7 pkgb
pkgb/__main__.py:7: error: Argument 1 to "this_is_a_typed_function" has incompatible type "int"; expected "unicode"
```
That above is expected, but when specifying the version in a config file.
```
$ echo -e "[mypy]\npython_version: 2.7" | mypy --config-file /dev/stdin pkgb
pkgb/__main__.py:1: error: Cannot find module named 'pkga'
```

For good measure, I included a python 2 print statement in the test package to ensure it is actually is running under python 2 mode ... [The example is included here](https://github.com/python/mypy/files/2383934/mypy5576.tar.gz). Extract the archive; in a Python 2 virtualenv, install the package in the `a` directory with `pip -e a`; run mypy on `pkgb`.

---

The behaviour seems to come from looking for typing information in different site-packages between the two cases. With `--python-version 2.7`, a Python 2 interpreter's site-packages are used. With the configuration file, the current interpreter's site-packages are used.

The pull request #5578 was an attempt to resolve this inconsistency by using the Python 2 site-packages in both configurations, but it seems like this broke some things for some folks.


Discussion:
Disclaimer: I haven't really been following this issue, so I may be missing some context, but I wonder if this is solvable by just having the people's whose config files broke with #5578 set the `python_executable` option in addition to the `python_version` option?
I believe in that case they would have to set `python_version` to 2.7 to check the Python 2 syntax and `python_executable` to a Python 3 interpreter to use its site-packages. There is something that ensures the versions here match and raises an exception otherwise. But maybe this should be allowed? I can't say for sure if mypy would behave as they'd like if it were allowed. That is interesting. 

On September 14, 2018 9:29:41 AM PDT, Michael Lee <notifications@github.com> wrote:
>Disclaimer: I haven't really been following this issue, so I may be
>missing some context, but I wonder if this is solvable by just having
>the people's whose config files broke with #5578 set the
>`python_executable` option in addition to the `python_version` option?
>
>-- 
>You are receiving this because you authored the thread.
>Reply to this email directly or view it on GitHub:
>https://github.com/python/mypy/issues/5620#issuecomment-421413079

> But maybe this should be allowed?

I think I will be fine with this solution (assuming it doesn't break anything else). I just want to have a clear solution for the scenario where PEP 561 stub packages are installed in an interpreter different from the one used to run the code that is being checked (for example in the CI server situation I mention in PR #5578).

@ethanhs I will be very grateful for your advice on this.
I think @Michael0x2a's solution is likely optimal here. I will work on a fix for this later.
Disclaimer: I haven't really been following this issue, so I may be missing some context, but I wonder if this is solvable by just having the people's whose config files broke with #5578 set the `python_executable` option in addition to the `python_version` option?
I believe in that case they would have to set `python_version` to 2.7 to check the Python 2 syntax and `python_executable` to a Python 3 interpreter to use its site-packages. There is something that ensures the versions here match and raises an exception otherwise. But maybe this should be allowed? I can't say for sure if mypy would behave as they'd like if it were allowed. That is interesting. 

On September 14, 2018 9:29:41 AM PDT, Michael Lee <notifications@github.com> wrote:
>Disclaimer: I haven't really been following this issue, so I may be
>missing some context, but I wonder if this is solvable by just having
>the people's whose config files broke with #5578 set the
>`python_executable` option in addition to the `python_version` option?
>
>-- 
>You are receiving this because you authored the thread.
>Reply to this email directly or view it on GitHub:
>https://github.com/python/mypy/issues/5620#issuecomment-421413079

> But maybe this should be allowed?

I think I will be fine with this solution (assuming it doesn't break anything else). I just want to have a clear solution for the scenario where PEP 561 stub packages are installed in an interpreter different from the one used to run the code that is being checked (for example in the CI server situation I mention in PR #5578).

@ethanhs I will be very grateful for your advice on this.
I think @Michael0x2a's solution is likely optimal here. I will work on a fix for this later.

Resolving patch:
diff --git a/mypy/main.py b/mypy/main.py
--- a/mypy/main.py
+++ b/mypy/main.py
@@ -227,17 +227,6 @@ def python_executable_prefix(v: str) -> List[str]:
         return ['python{}'.format(v)]
 
 
-def _python_version_from_executable(python_executable: str) -> Tuple[int, int]:
-    try:
-        check = subprocess.check_output([python_executable, '-c',
-                                         'import sys; print(repr(sys.version_info[:2]))'],
-                                        stderr=subprocess.STDOUT).decode()
-        return ast.literal_eval(check)
-    except (subprocess.CalledProcessError, FileNotFoundError):
-        raise PythonExecutableInferenceError(
-            'invalid Python executable {}'.format(python_executable))
-
-
 def _python_executable_from_version(python_version: Tuple[int, int]) -> str:
     if sys.version_info[:2] == python_version:
         return sys.executable
@@ -253,37 +242,25 @@ def _python_executable_from_version(python_version: Tuple[int, int]) -> str:
             ' perhaps try --python-executable, or --no-site-packages?'.format(python_version))
 
 
-def infer_python_version_and_executable(options: Options,
-                                        special_opts: argparse.Namespace) -> None:
-    """Infer the Python version or executable from each other. Check they are consistent.
+def infer_python_executable(options: Options,
+                            special_opts: argparse.Namespace) -> None:
+    """Infer the Python executable from the given version.
 
-    This function mutates options based on special_opts to infer the correct Python version and
-    executable to use.
+    This function mutates options based on special_opts to infer the correct Python executable
+    to use.
     """
-    # Infer Python version and/or executable if one is not given
-
     # TODO: (ethanhs) Look at folding these checks and the site packages subprocess calls into
     # one subprocess call for speed.
-    if special_opts.python_executable is not None and special_opts.python_version is not None:
-        py_exe_ver = _python_version_from_executable(special_opts.python_executable)
-        if py_exe_ver != special_opts.python_version:
-            raise PythonExecutableInferenceError(
-                'Python version {} did not match executable {}, got version {}.'.format(
-                    special_opts.python_version, special_opts.python_executable, py_exe_ver
-                ))
-        else:
-            options.python_version = special_opts.python_version
-            options.python_executable = special_opts.python_executable
-    elif special_opts.python_executable is None and special_opts.python_version is not None:
-        options.python_version = special_opts.python_version
-        py_exe = None
+
+    # Use the command line specified executable, or fall back to one set in the
+    # config file. If an executable is not specified, infer it from the version
+    # (unless no_executable is set)
+    python_executable = special_opts.python_executable or options.python_executable
+
+    if python_executable is None:
         if not special_opts.no_executable:
-            py_exe = _python_executable_from_version(special_opts.python_version)
-        options.python_executable = py_exe
-    elif special_opts.python_version is None and special_opts.python_executable is not None:
-        options.python_version = _python_version_from_executable(
-            special_opts.python_executable)
-        options.python_executable = special_opts.python_executable
+            python_executable = _python_executable_from_version(options.python_version)
+    options.python_executable = python_executable
 
 
 HEADER = """%(prog)s [-h] [-v] [-V] [more options; see below]
@@ -744,8 +721,11 @@ def add_invertible_flag(flag: str,
         print("Warning: --quick-and-dirty is deprecated.  It will disappear in the next release.",
               file=sys.stderr)
 
+    # The python_version is either the default, which can be overridden via a config file,
+    # or stored in special_opts and is passed via the command line.
+    options.python_version = special_opts.python_version or options.python_version
     try:
-        infer_python_version_and_executable(options, special_opts)
+        infer_python_executable(options, special_opts)
     except PythonExecutableInferenceError as e:
         parser.error(str(e))
 


Test patch:
diff --git a/mypy/test/testargs.py b/mypy/test/testargs.py
--- a/mypy/test/testargs.py
+++ b/mypy/test/testargs.py
@@ -12,7 +12,7 @@
 from mypy.test.helpers import Suite, assert_equal
 from mypy.options import Options
 from mypy.main import (process_options, PythonExecutableInferenceError,
-                       infer_python_version_and_executable)
+                       infer_python_executable)
 
 
 class ArgSuite(Suite):
@@ -47,22 +47,30 @@ def test_executable_inference(self) -> None:
         assert options.python_version == sys.version_info[:2]
         assert options.python_executable == sys.executable
 
-        # test that we error if the version mismatch
-        # argparse sys.exits on a parser.error, we need to check the raw inference function
-        options = Options()
-
-        special_opts = argparse.Namespace()
-        special_opts.python_executable = sys.executable
-        special_opts.python_version = (2, 10)  # obviously wrong
-        special_opts.no_executable = None
-        with pytest.raises(PythonExecutableInferenceError) as e:
-            infer_python_version_and_executable(options, special_opts)
-        assert str(e.value) == 'Python version (2, 10) did not match executable {}, got' \
-                               ' version {}.'.format(sys.executable, sys.version_info[:2])
-
         # test that --no-site-packages will disable executable inference
         matching_version = base + ['--python-version={}'.format(sys_ver_str),
                                    '--no-site-packages']
         _, options = process_options(matching_version)
         assert options.python_version == sys.version_info[:2]
         assert options.python_executable is None
+
+        # Test setting python_version/executable from config file
+        special_opts = argparse.Namespace()
+        special_opts.python_executable = None
+        special_opts.python_version = None
+        special_opts.no_executable = None
+
+        # first test inferring executable from version
+        options = Options()
+        options.python_executable = None
+        options.python_version = sys.version_info[:2]
+        infer_python_executable(options, special_opts)
+        assert options.python_version == sys.version_info[:2]
+        assert options.python_executable == sys.executable
+
+        # then test inferring version from executable
+        options = Options()
+        options.python_executable = sys.executable
+        infer_python_executable(options, special_opts)
+        assert options.python_version == sys.version_info[:2]
+        assert options.python_executable == sys.executable