internlm/SWE-Fixer-Train-110K huggingface dataset:
Repository: python/mypy

Issue:
Overlapping entries in mypy.ini are processed in nondeterministic order
Suppose you have these entries in `mypy.ini`:
```
[mypy-metaserver.lib.shared_folder.*]
ignore_errors = True
[mypy-metaserver.lib.shared_folder.perms.*]
ignore_errors = False
```
For a submodule of package `metaserver.lib.shared_folder.perms` it will be nondeterministic whether that file is processed with `ignore_errors = True` or `ignore_errors = False`.

The reason is that `Options.clone_for_module()` in `mypy/options.py` loops over `self.per_module_options` which is a dict, and dict ordering is randomized by default in Python 3 (unless `PYTHONHASHSEED` is set).


Discussion:
Fortunate the solution seems straightforward, we just need to use OrderedDict to hold `per_module_options`.

Resolving patch:
diff --git a/mypy/options.py b/mypy/options.py
--- a/mypy/options.py
+++ b/mypy/options.py
@@ -1,8 +1,9 @@
+from collections import OrderedDict
 import fnmatch
 import pprint
 import sys
 
-from typing import Mapping, Optional, Tuple, List, Pattern, Dict
+from typing import Mapping, MutableMapping, Optional, Tuple, List, Pattern, Dict
 
 from mypy import defaults
 
@@ -119,7 +120,8 @@ def __init__(self) -> None:
         self.plugins = []  # type: List[str]
 
         # Per-module options (raw)
-        self.per_module_options = {}  # type: Dict[Pattern[str], Dict[str, object]]
+        pm_opts = OrderedDict()  # type: OrderedDict[Pattern[str], Dict[str, object]]
+        self.per_module_options = pm_opts
 
         # -- development options --
         self.verbosity = 0  # More verbose messages (for troubleshooting)


Test patch:
diff --git a/test-data/unit/cmdline.test b/test-data/unit/cmdline.test
--- a/test-data/unit/cmdline.test
+++ b/test-data/unit/cmdline.test
@@ -1031,3 +1031,26 @@ m.py:4: error: Missing type parameters for generic type
 m.py:5: error: Missing type parameters for generic type
 m.py:6: error: Missing type parameters for generic type
 m.py:7: error: Missing type parameters for generic type
+
+[case testDeterministicSectionOrdering]
+# cmd: mypy a
+[file a/__init__.py]
+[file a/b/__init__.py]
+[file a/b/c/__init__.py]
+[file a/b/c/d/__init__.py]
+[file a/b/c/d/e/__init__.py]
+0()
+[file mypy.ini]
+[[mypy]
+[[mypy-a.*]
+ignore_errors = True
+[[mypy-a.b.*]
+ignore_errors = True
+[[mypy-a.b.c.*]
+ignore_errors = True
+[[mypy-a.b.c.d.*]
+ignore_errors = True
+[[mypy-a.b.c.d.e]
+ignore_errors = False
+[out]
+a/b/c/d/e/__init__.py:1: error: "int" not callable
Repository: python/mypy

Issue:
variables starting with an underscore are not imported with * imports even when present in __all__
Reduced example of the issue:

defs.py:
```python
def __pragma__(arg: str) -> None:
    pass


__all__ = ["__pragma__"]
```
main.py:
```python
from defs import *

__pragma__('main')

print('hi')
```
The current mypy behavior differs as follows:
```
$ python main.py
hi
$ mypy main.py
main.py:3: error: Name '__pragma__' is not defined
```

This is an most likely an obscure case, but if it could be added to `mypy` I would greatly appreciate it that would be greatly appreciated. I can also write a pull request to implement this, if it's deemed worth adding.

My use case for this is providing stubs for 'meta' functions provided by [transcrypt](http://transcrypt.com/), a Python transpiler/pre-processor. Transcrypt provides `__pragma__` and `__new__` to control / augment the transpilation. Having `import * from defs` work to import both these transcrypt-builtins and all other transpilation-builtin types would greatly increase usability of mypy and transcrypt together.


Discussion:

Resolving patch:
diff --git a/mypy/semanal.py b/mypy/semanal.py
--- a/mypy/semanal.py
+++ b/mypy/semanal.py
@@ -1483,7 +1483,9 @@ def visit_import_all(self, i: ImportAll) -> None:
             self.add_submodules_to_parent_modules(i_id, True)
             for name, node in m.names.items():
                 node = self.normalize_type_alias(node, i)
-                if not name.startswith('_') and node.module_public:
+                # if '__all__' exists, all nodes not included have had module_public set to
+                # False, and we can skip checking '_' because it's been explicitly included.
+                if node.module_public and (not name.startswith('_') or '__all__' in m.names):
                     existing_symbol = self.globals.get(name)
                     if existing_symbol:
                         # Import can redefine a variable. They get special treatment.


Test patch:
diff --git a/test-data/unit/check-modules.test b/test-data/unit/check-modules.test
--- a/test-data/unit/check-modules.test
+++ b/test-data/unit/check-modules.test
@@ -427,6 +427,25 @@ __all__ = [u'a', u'b', u'c']
 
 [out]
 
+[case testUnderscoreExportedValuesInImportAll]
+import typing
+from m import *
+_ = a
+_ = _b
+_ = __c__
+_ = ___d
+_ = e
+_ = f # E: Name 'f' is not defined
+_ = _g # E: Name '_g' is not defined
+[file m.py]
+__all__ = ['a']
+__all__ += ('_b',)
+__all__.append('__c__')
+__all__.extend(('___d', 'e'))
+
+a = _b = __c__ = ___d = e = f = _g = 1
+[builtins fixtures/module_all.pyi]
+
 [case testEllipsisInitializerInStubFileWithType]
 import m
 m.x = '' # E: Incompatible types in assignment (expression has type "str", variable has type "int")
Repository: python/mypy

Issue:
If isinstance target type is Any, narrow down to Any
Currently if the second argument to `isinstance` has type `Any`, we don't narrow the type down. This doesn't work as expected with ignored imports. Example (assume that `foo` has a missing stub):

```py
from typing import Union

from foo import A  # type: ignore

def f(x: Union[A, str]) -> None:
    if isinstance(x, A):
        x.method_only_in_a()  # Item "str" of "Union[Any, str]" has no attribute "method_only_in_a"
```

I'd expect the type of `x` to be 'narrowed' (actually widened) to `Any`, and mypy should probably not generate an error.


Discussion:

Resolving patch:
diff --git a/mypy/checker.py b/mypy/checker.py
--- a/mypy/checker.py
+++ b/mypy/checker.py
@@ -3230,6 +3230,8 @@ def get_isinstance_type(expr: Expression,
         elif isinstance(typ, Instance) and typ.type.fullname() == 'builtins.type':
             object_type = Instance(typ.type.mro[-1], [])
             types.append(TypeRange(object_type, is_upper_bound=True))
+        elif isinstance(typ, AnyType):
+            types.append(TypeRange(typ, is_upper_bound=False))
         else:  # we didn't see an actual type, but rather a variable whose value is unknown to us
             return None
     if not types:


Test patch:
diff --git a/test-data/unit/check-isinstance.test b/test-data/unit/check-isinstance.test
--- a/test-data/unit/check-isinstance.test
+++ b/test-data/unit/check-isinstance.test
@@ -1975,3 +1975,33 @@ def f() -> None:
 [typing fixtures/typing-full.pyi]
 [builtins fixtures/dict.pyi]
 [out]
+
+[case testIsinstanceWidensWithAnyArg]
+from typing import Any
+class A: ...
+B: Any
+x: A
+x.foo()  # E: "A" has no attribute "foo"
+assert isinstance(x, B)
+x.foo()
+reveal_type(x)  # E: Revealed type is 'Any'
+[builtins fixtures/isinstance.pyi]
+
+[case testIsinstanceWidensUnionWithAnyArg]
+from typing import Any, Union
+class A: ...
+B: Any
+x: Union[A, B]
+reveal_type(x)  # E: Revealed type is 'Union[__main__.A, Any]'
+assert isinstance(x, B)
+reveal_type(x)  # E: Revealed type is 'Any'
+[builtins fixtures/isinstance.pyi]
+
+[case testIsinstanceIgnoredImport]
+from typing import Union
+from foo import A  # type: ignore
+def f(x: Union[A, str]) -> None:
+    x.method_only_in_a()  # E: Item "str" of "Union[Any, str]" has no attribute "method_only_in_a"
+    if isinstance(x, A):
+        x.method_only_in_a()
+[builtins fixtures/isinstance.pyi]
Repository: python/mypy

Issue:
Regression - filtering `eval` and `cmd` tests
My own #3870 introduced a bug - running `./runtests.py eval` or `./runtests cmd` does not work correctly.


Discussion:

Resolving patch:
diff --git a/runtests.py b/runtests.py
--- a/runtests.py
+++ b/runtests.py
@@ -89,17 +89,17 @@ def add_mypy_package(self, name: str, packagename: str, *flags: str) -> None:
     def add_mypy_string(self, name: str, *args: str, cwd: Optional[str] = None) -> None:
         self.add_mypy_cmd(name, ['-c'] + list(args), cwd=cwd)
 
-    def add_pytest(self, name: str, pytest_files: List[str], coverage: bool = True) -> None:
-        pytest_args = pytest_files + self.arglist + self.pyt_arglist
-        full_name = 'pytest %s' % name
-        if not self.allow(full_name):
+    def add_pytest(self, pytest_files: List[str], coverage: bool = True) -> None:
+        pytest_files = [name for name in pytest_files if self.allow(name[4:])]
+        if not pytest_files:
             return
+        pytest_args = pytest_files + self.arglist + self.pyt_arglist
         if coverage and self.coverage:
             args = [sys.executable, '-m', 'pytest', '--cov=mypy'] + pytest_args
         else:
             args = [sys.executable, '-m', 'pytest'] + pytest_args
 
-        self.waiter.add(LazySubprocess(full_name, args, env=self.env, passthrough=self.verbosity),
+        self.waiter.add(LazySubprocess('pytest', args, env=self.env, passthrough=self.verbosity),
                         sequential=True)
 
     def add_python(self, name: str, *args: str, cwd: Optional[str] = None) -> None:
@@ -233,7 +233,7 @@ def test_path(*names: str):
 
 
 def add_pytest(driver: Driver) -> None:
-    driver.add_pytest('pytest', PYTEST_FILES)
+    driver.add_pytest(PYTEST_FILES)
 
 
 def add_myunit(driver: Driver) -> None:


Test patch:
Repository: python/mypy

Issue:
Spurious "No overload variant matches argument types"
I have an overloaded function that instantiates a class:

```python
from typing import Any, Type, TypeVar, overload
T = TypeVar('T')

@overload
def make(cls: Type[T]) -> T:
    pass

@overload
def make() -> Any:
    pass

def make(cls=None):
    if cls:
        return cls()
    return object()

c = make(int)

reveal_type(c)
```

Type checking with mypy (using latest from master branch) results in the error:

```
[out]
a.py:17: error: No overload variant of "make" matches argument types [Overload(def (x: typing.SupportsInt =) -> builtins.int, def (x: Union[builtins.str, builtins.bytes], base: builtins.int =) -> builtins.int)]
a.py:19: error: Revealed type is 'Any'
```

Removing the overload prevents the error:


```python
from typing import Type, TypeVar
T = TypeVar('T')

def make(cls: Type[T]) -> T:
    return cls()

c = make(int)
```

Is this a bug or user error?

thanks!



Discussion:
Looks like a bug to me. I think it may be translating the type `int` to a callable incorrectly, so it no longer matches `Type`.
``Overloaded`` actual for ``Type[...]``formal was just overlooked when implementing ``Type[...]``. This would be essentially just a two-line fix.

Resolving patch:
diff --git a/mypy/checkexpr.py b/mypy/checkexpr.py
--- a/mypy/checkexpr.py
+++ b/mypy/checkexpr.py
@@ -2817,10 +2817,10 @@ def overload_arg_similarity(actual: Type, formal: Type) -> int:
             # Since Type[T] is covariant, check if actual = Type[A] is
             # a subtype of formal = Type[F].
             return overload_arg_similarity(actual.item, formal.item)
-        elif isinstance(actual, CallableType) and actual.is_type_obj():
+        elif isinstance(actual, FunctionLike) and actual.is_type_obj():
             # Check if the actual is a constructor of some sort.
             # Note that this is this unsound, since we don't check the __init__ signature.
-            return overload_arg_similarity(actual.ret_type, formal.item)
+            return overload_arg_similarity(actual.items()[0].ret_type, formal.item)
         else:
             return 0
     if isinstance(actual, TypedDictType):


Test patch:
diff --git a/test-data/unit/check-overloading.test b/test-data/unit/check-overloading.test
--- a/test-data/unit/check-overloading.test
+++ b/test-data/unit/check-overloading.test
@@ -1272,3 +1272,28 @@ a: Any
 # The return type is not ambiguous so Any arguments cause no ambiguity.
 reveal_type(f(a, 1, 1)) # E: Revealed type is 'builtins.str'
 reveal_type(f(1, *a))  # E: Revealed type is 'builtins.str'
+
+[case testOverloadOnOverloadWithType]
+from typing import Any, Type, TypeVar, overload
+from mod import MyInt
+T = TypeVar('T')
+
+@overload
+def make(cls: Type[T]) -> T: pass
+@overload
+def make() -> Any: pass
+
+def make(*args):
+    pass
+
+c = make(MyInt)
+reveal_type(c) # E: Revealed type is 'mod.MyInt*'
+
+[file mod.pyi]
+from typing import overload
+class MyInt:
+    @overload
+    def __init__(self, x: str) -> None: pass
+    @overload
+    def __init__(self, x: str, y: int) -> None: pass
+[out]
Repository: python/mypy

Issue:
Skip Py2 subprocess tests if Py2 interpreter found does not have typing installed
The behavior of mypy with regard to Python 2.7 tests is inconsistent. On the one hand, if no Python 2.7 interpreter can be found at all, https://github.com/python/mypy/blob/master/mypy/util.py#L66 will return `None` and mypy will happily skip the Py2 tests. But if a Python 2.7 interpreter is found, but is missing the `typing` backport (which mypy no longer bundles), mypy will go ahead and run the Py2 tests and they will fail.

Our build boxes are in the midst of an upgrade from an OS version with Python 2.6 as system Python to one with Python 2.7. On the older OS, mypy builds fine and just skips the Py2 tests (which is fine). On the newer OS, it finds the system Python 2.7 and those tests fail for lack of `typing`. It'sdifficult for us to install the `typing` module into the system Python on these hosts.

I propose that it would be more consistent if mypy considers "no Python 2.7 with typing available" equivalent to "no Python 2.7 available" and skips the Py2 tests. It would be quite easy to add a `python -c "import typing"` check to `try_find_python2_interpreter`. I can submit a PR for this, if the idea is approved.


Discussion:
Sure, that sounds reasonable -- go ahead and send a PR.
Actually, this specific issue will no longer be a problem if https://github.com/python/mypy/pull/4025 is merged as-is. We will no longer require the runtime `typing` to be there. Furthermore, that change *should* remove our dependency on a runtime Python 2 all together, as we don't test evaluation of Python anymore. 

Resolving patch:
diff --git a/mypy/util.py b/mypy/util.py
--- a/mypy/util.py
+++ b/mypy/util.py
@@ -69,10 +69,11 @@ def try_find_python2_interpreter() -> Optional[str]:
         return _python2_interpreter
     for interpreter in default_python2_interpreter:
         try:
-            process = subprocess.Popen([interpreter, '-V'], stdout=subprocess.PIPE,
-                                       stderr=subprocess.STDOUT)
-            stdout, stderr = process.communicate()
-            if b'Python 2.7' in stdout:
+            retcode = subprocess.Popen([
+                interpreter, '-c',
+                'import sys, typing; assert sys.version_info[:2] == (2, 7)'
+            ]).wait()
+            if not retcode:
                 _python2_interpreter = interpreter
                 return interpreter
         except OSError:


Test patch:
Repository: python/mypy

Issue:
SupportsAbs not compatible with NamedTuple
```
from typing import SupportsAbs, NamedTuple

class Point(NamedTuple('Point', [('x', int), ('y', int)]), SupportsAbs[int]):
    def __abs__(p) -> int:
        return abs(p.x) + abs(p.y)

def test(a: Point) -> bool:
    return abs(a) == 2  # E: Argument 1 to "abs" has incompatible type "Point"; expected SupportsAbs[None]
```

There's no error in `test` if the `NamedTuple` superclass of `Point` is removed.



Discussion:

Resolving patch:
diff --git a/mypy/constraints.py b/mypy/constraints.py
--- a/mypy/constraints.py
+++ b/mypy/constraints.py
@@ -373,12 +373,8 @@ def visit_instance(self, template: Instance) -> List[Constraint]:
                 cb = infer_constraints(template.args[0], item, SUPERTYPE_OF)
                 res.extend(cb)
             return res
-        elif (isinstance(actual, TupleType) and template.type.is_protocol and
-              self.direction == SUPERTYPE_OF):
-            if mypy.subtypes.is_subtype(actual.fallback, erase_typevars(template)):
-                res.extend(infer_constraints(template, actual.fallback, self.direction))
-                return res
-            return []
+        elif isinstance(actual, TupleType) and self.direction == SUPERTYPE_OF:
+            return infer_constraints(template, actual.fallback, self.direction)
         else:
             return []
 


Test patch:
diff --git a/test-data/unit/check-tuples.test b/test-data/unit/check-tuples.test
--- a/test-data/unit/check-tuples.test
+++ b/test-data/unit/check-tuples.test
@@ -941,6 +941,21 @@ reveal_type(t[x:])  # E: Revealed type is 'Union[builtins.int, builtins.str]'
 t[y:]  # E: Slice index must be an integer or None
 [builtins fixtures/tuple.pyi]
 
+[case testInferTupleTypeFallbackAgainstInstance]
+from typing import TypeVar, Generic, Tuple
+T = TypeVar('T')
+
+class Base(Generic[T]): pass
+def f(x: Base[T]) -> T: pass
+
+class DT(Tuple[str, str], Base[int]):
+    pass
+
+reveal_type(f(DT())) # E: Revealed type is 'builtins.int*'
+
+[builtins fixtures/tuple.pyi]
+[out]
+
 [case testTypeTupleClassmethod]
 from typing import Tuple, Type
 
diff --git a/test-data/unit/pythoneval.test b/test-data/unit/pythoneval.test
--- a/test-data/unit/pythoneval.test
+++ b/test-data/unit/pythoneval.test
@@ -1418,6 +1418,17 @@ async def main() -> None:
 _testAsyncioGatherPreciseType.py:9: error: Revealed type is 'builtins.str'
 _testAsyncioGatherPreciseType.py:10: error: Revealed type is 'builtins.str'
 
+[case testMultipleInheritanceWorksWithTupleTypeGeneric]
+from typing import SupportsAbs, NamedTuple
+
+class Point(NamedTuple('Point', [('x', int), ('y', int)]), SupportsAbs[int]):
+    def __abs__(p) -> int:
+        return abs(p.x) + abs(p.y)
+
+def test(a: Point) -> bool:
+    return abs(a) == 2
+[out]
+
 [case testNoCrashOnGenericUnionUnpacking]
 from typing import Union, Dict
Repository: python/mypy

Issue:
No constraint inferred for Any vs. Type[T]
I'd expect the revealed type to be `Any` instead of `<nothing>` for this example:

```py
from typing import Type, Any, TypeVar

T = TypeVar('T')

def f(c: Type[T]) -> T: ...

x: Any
reveal_type(f(x))  # <nothing>
```

It looks like constraint inference doesn't work correctly when matching `Any` against `Type[T]`.


Discussion:

Resolving patch:
diff --git a/mypy/constraints.py b/mypy/constraints.py
--- a/mypy/constraints.py
+++ b/mypy/constraints.py
@@ -506,6 +506,8 @@ def visit_type_type(self, template: TypeType) -> List[Constraint]:
                                      self.direction)
         elif isinstance(self.actual, TypeType):
             return infer_constraints(template.item, self.actual.item, self.direction)
+        elif isinstance(self.actual, AnyType):
+            return infer_constraints(template.item, self.actual, self.direction)
         else:
             return []
 


Test patch:
diff --git a/test-data/unit/check-generics.test b/test-data/unit/check-generics.test
--- a/test-data/unit/check-generics.test
+++ b/test-data/unit/check-generics.test
@@ -1747,3 +1747,13 @@ class D:
     def __or__(self, x: G[X]) -> G[X]: pass
     def __ior__(self, x: G[S2]) -> G[S2]: pass \
         # E: Signatures of "__ior__" and "__or__" are incompatible
+
+[case testConstraintInferenceForAnyAgainstTypeT]
+from typing import Type, Any, TypeVar
+
+T = TypeVar('T')
+
+def f(c: Type[T]) -> T: ...
+
+x: Any
+reveal_type(f(x))  # E: Revealed type is 'Any'
Repository: python/mypy

Issue:
Virtualenv install broken on Windows
It seems that virtualenvs do not have a userbase prefix, so my recent changes in #4005 broke this. First reported [here](https://github.com/python/mypy/pull/4005#issuecomment-337577401).


Discussion:

Resolving patch:
diff --git a/mypy/build.py b/mypy/build.py
--- a/mypy/build.py
+++ b/mypy/build.py
@@ -13,6 +13,7 @@
 import binascii
 import collections
 import contextlib
+from distutils.sysconfig import get_python_lib
 import gc
 import hashlib
 import json
@@ -229,7 +230,12 @@ def default_data_dir(bin_dir: Optional[str]) -> str:
     """
     if not bin_dir:
         if os.name == 'nt':
-            prefixes = [os.path.join(sys.prefix, 'Lib'), os.path.join(site.getuserbase(), 'lib')]
+            prefixes = [os.path.join(sys.prefix, 'Lib')]
+            try:
+                prefixes.append(os.path.join(site.getuserbase(), 'lib'))
+            except AttributeError:
+                # getuserbase in not available in virtualenvs
+                prefixes.append(os.path.join(get_python_lib(), 'lib'))
             for parent in prefixes:
                     data_dir = os.path.join(parent, 'mypy')
                     if os.path.exists(data_dir):


Test patch:
Repository: python/mypy

Issue:
Don't complain about abstract attributes if class has an Any base class
Mypy complains about abstract method `C.f` even though `A` may well implement it:

```py
from typing import Any
from abc import abstractmethod, ABCMeta

A: Any

class D(metaclass=ABCMeta):
    @abstractmethod
    def f(self) -> None: pass

class C(A, D):
    pass

C() # Cannot instantiate abstract class 'C' with abstract attribute 'f'
```

If a class has an `Any` base class, mypy shouldn't complain about any abstract attributes.


Discussion:

Resolving patch:
diff --git a/mypy/checkexpr.py b/mypy/checkexpr.py
--- a/mypy/checkexpr.py
+++ b/mypy/checkexpr.py
@@ -543,7 +543,8 @@ def check_call(self, callee: Type, args: List[Expression],
 
             if (callee.is_type_obj() and callee.type_object().is_abstract
                     # Exceptions for Type[...] and classmethod first argument
-                    and not callee.from_type_type and not callee.is_classmethod_class):
+                    and not callee.from_type_type and not callee.is_classmethod_class
+                    and not callee.type_object().fallback_to_any):
                 type = callee.type_object()
                 self.msg.cannot_instantiate_abstract_class(
                     callee.type_object().name(), type.abstract_attributes,


Test patch:
diff --git a/test-data/unit/check-abstract.test b/test-data/unit/check-abstract.test
--- a/test-data/unit/check-abstract.test
+++ b/test-data/unit/check-abstract.test
@@ -638,6 +638,21 @@ class C:
     def __add__(self, other: int) -> B: pass
 [out]
 
+[case testAbstractClassWithAnyBase]
+from typing import Any
+from abc import abstractmethod, ABCMeta
+
+A: Any
+
+class D(metaclass=ABCMeta):
+    @abstractmethod
+    def f(self) -> None: pass
+
+class C(A, D):
+    pass
+
+C()  # A might implement 'f'
+
 
 -- Abstract properties
 -- -------------------