internlm/SWE-Fixer-Train-110K huggingface dataset:
Repository: python/mypy

Issue:
mypy gets confused by double coding cookie
When a file has two different coding cookies (one in line 1, one in line 2) mypy gets confused and uses the cookie from the second line. It should use the first.

Low priority; I found this by accident and the new parser will presumably make this go away.



Discussion:
I remember reading through [the PEP](https://www.python.org/dev/peps/pep-0263/) and [the docs](https://docs.python.org/3/howto/unicode.html#unicode-literals-in-python-source-code) about this when I was considering picking up #522. I was slightly surprised that neither of them seem to document the expected behaviour when both the first and second lines contain a valid encoding comment. However it's entirely possible that I missed something, and in any case it's probably best to match the behaviour of CPython as you suggest.

FWIW I clarified this in the PEP.


Resolving patch:
diff --git a/mypy/util.py b/mypy/util.py
--- a/mypy/util.py
+++ b/mypy/util.py
@@ -7,7 +7,7 @@
 
 T = TypeVar('T')
 
-ENCODING_RE = re.compile(br'(\s*#.*(\r\n?|\n))?\s*#.*coding[:=]\s*([-\w.]+)')
+ENCODING_RE = re.compile(br'(\s*#.*(\r\n?|\n)){0,1}?\s*#.*coding[:=]\s*([-\w.]+)')
 
 default_python2_interpreter = ['python2', 'python', '/usr/bin/python']
 


Test patch:
diff --git a/mypy/test/data/parse-errors.test b/mypy/test/data/parse-errors.test
--- a/mypy/test/data/parse-errors.test
+++ b/mypy/test/data/parse-errors.test
@@ -453,12 +453,34 @@ file:2: error: Unknown encoding 'uft-8'
 [out]
 file:1: error: Unknown encoding 'Uft.8'
 
-[case testInvalidEncoding2]
+[case testInvalidEncoding3]
 #!/usr/bin python
 # vim: set fileencoding=uft8 :
 [out]
 file:2: error: Unknown encoding 'uft8'
 
+[case testDoubleEncoding]
+# coding: uft8
+# coding: utf8
+# The first coding cookie should be used and fail.
+[out]
+file:1: error: Unknown encoding 'uft8'
+
+[case testDoubleEncoding2]
+# Again the first cookie should be used and fail.
+# coding: uft8
+# coding: utf8
+[out]
+file:2: error: Unknown encoding 'uft8'
+
+[case testDoubleEncoding3]
+# If the third line were interpreted as a coding cookie, we'd get a
+# different error message.
+# coding: ascii
+á = 1
+[out]
+file:4: error: Unrecognized character
+
 [case testLongLiteralInPython3]
 2L
 0x2L
Repository: python/mypy

Issue:
The new contextlib stubs trigger an incorrect "Invalid method type" error
Repro without contextlib:

```
from typing import Callable
def cm(func) -> Callable[..., None]: pass
class C:
    @cm
    def foo(self) -> None: pass
C().foo()  # E: Invalid method type
```

First spotted in https://github.com/python/typeshed/pull/122



Discussion:

Resolving patch:
diff --git a/mypy/checkmember.py b/mypy/checkmember.py
--- a/mypy/checkmember.py
+++ b/mypy/checkmember.py
@@ -231,8 +231,8 @@ def lookup_member_var_or_accessor(info: TypeInfo, name: str,
 def check_method_type(functype: FunctionLike, itype: Instance,
                       context: Context, msg: MessageBuilder) -> None:
     for item in functype.items():
-        if not item.arg_types or item.arg_kinds[0] != ARG_POS:
-            # No positional first (self) argument.
+        if not item.arg_types or item.arg_kinds[0] not in (ARG_POS, ARG_STAR):
+            # No positional first (self) argument (*args is okay).
             msg.invalid_method_type(item, context)
         else:
             # Check that self argument has type 'Any' or valid instance type.


Test patch:
diff --git a/mypy/test/data/check-varargs.test b/mypy/test/data/check-varargs.test
--- a/mypy/test/data/check-varargs.test
+++ b/mypy/test/data/check-varargs.test
@@ -557,3 +557,15 @@ x = f
 x = g # E: Incompatible types in assignment (expression has type Callable[[str], None], variable has type Callable[[int], None])
 [builtins fixtures/list.py]
 [out]
+
+
+-- Decorated method where self is implied by *args
+-- -----------------------------------------------
+
+[case testVarArgsCallableSelf]
+from typing import Callable
+def cm(func) -> Callable[..., None]: pass
+class C:
+    @cm
+    def foo(self) -> None: pass
+C().foo()
Repository: python/mypy

Issue:
assert and isinstance type inference
This code should be accepted by mypy:

``` python
def f(x: Union[int, str]) -> None:
    assert isinstance(x, int)
    x + 1  # type of x is int, so this should be okay
```

The type type checker should know about type assertions.



Discussion:
This could sometimes be used as an arguably more Pythonic replacement for casts.

I found a bunch of occurrences of this in our server code; bumping priority to 0.3.2.


Resolving patch:
diff --git a/mypy/checker.py b/mypy/checker.py
--- a/mypy/checker.py
+++ b/mypy/checker.py
@@ -1709,6 +1709,16 @@ def visit_operator_assignment_stmt(self,
     def visit_assert_stmt(self, s: AssertStmt) -> Type:
         self.accept(s.expr)
 
+        # If this is asserting some isinstance check, bind that type in the following code
+        true_map, _ = find_isinstance_check(
+            s.expr, self.type_map,
+            self.typing_mode_weak()
+        )
+
+        if true_map:
+            for var, type in true_map.items():
+                self.binder.push(var, type)
+
     def visit_raise_stmt(self, s: RaiseStmt) -> Type:
         """Type check a raise statement."""
         self.breaking_out = True


Test patch:
diff --git a/mypy/test/data/check-isinstance.test b/mypy/test/data/check-isinstance.test
--- a/mypy/test/data/check-isinstance.test
+++ b/mypy/test/data/check-isinstance.test
@@ -817,3 +817,10 @@ def f(x: Union[A, B]) -> None:
     if not isinstance(x, B):
         f(x)
 [builtins fixtures/isinstance.py]
+
+[case testAssertIsinstance]
+def f(x: object):
+    assert isinstance(x, int)
+    y = 0 # type: int
+    y = x
+[builtins fixtures/isinstance.py]
Repository: python/mypy

Issue:
unknown variable not caught in `dict` constructor
A file with just

``` python
x = dict(undefined_variable)
```

succeeds, when it should give a `Name 'undefined_variable' is not defined` error.  Without the `dict` constructor, the error is generated correctly.



Discussion:
And here's why: https://github.com/python/mypy/blob/master/mypy/semanal.py#L1813

Decided to fix this, in case it had a bearing on the more complex issue we were investigating yesterday (though I think it doesn't, actually).


Resolving patch:
diff --git a/mypy/semanal.py b/mypy/semanal.py
--- a/mypy/semanal.py
+++ b/mypy/semanal.py
@@ -1811,6 +1811,9 @@ def translate_dict_call(self, call: CallExpr) -> Optional[DictExpr]:
         if not call.args:
             return None
         if not all(kind == ARG_NAMED for kind in call.arg_kinds):
+            # Must still accept those args.
+            for a in call.args:
+                a.accept(self)
             return None
         expr = DictExpr([(StrExpr(key), value)
                          for key, value in zip(call.arg_names, call.args)])


Test patch:
diff --git a/mypy/test/data/check-expressions.test b/mypy/test/data/check-expressions.test
--- a/mypy/test/data/check-expressions.test
+++ b/mypy/test/data/check-expressions.test
@@ -1376,3 +1376,9 @@ d4 = dict(a=1, b=1)
 d4.xyz # E: Dict[str, int] has no attribute "xyz"
 d5 = dict(a=1, b='') # type: Dict[str, Any]
 [builtins fixtures/dict.py]
+
+[case testDictWithoutKeywordArgs]
+dict(undefined)
+[builtins fixtures/dict.py]
+[out]
+main:1: error: Name 'undefined' is not defined
diff --git a/mypy/test/data/fixtures/dict.py b/mypy/test/data/fixtures/dict.py
--- a/mypy/test/data/fixtures/dict.py
+++ b/mypy/test/data/fixtures/dict.py
@@ -1,6 +1,6 @@
 # Builtins stub used in dictionary-related test cases.
 
-from typing import TypeVar, Generic, Iterable, Iterator
+from typing import TypeVar, Generic, Iterable, Iterator, Any
 
 T = TypeVar('T')
 S = TypeVar('S')
@@ -11,6 +11,7 @@ def __init__(self) -> None: pass
 class type: pass
 
 class dict(Generic[T, S]):
+    def __init__(self, arg: Any = None) -> None: pass
     def __setitem__(self, k: T, v: S) -> None: pass
     def update(self, a: 'dict[T, S]') -> None: pass
 class int: pass # for convenience
Repository: python/mypy

Issue:
Fails with `--silent-imports`; passes without it
```
$ mypy -f tornado/ioloop.py
$ mypy -f -s tornado/ioloop.py
tornado/ioloop.py:48: error: Module has no attribute 'set_close_exec'
tornado/ioloop.py:48: error: Module has no attribute 'Waker'
```

In Tornado, I prefer to use inline annotations instead of stub files, but occasionally I use a stub for something that would be too tricky to mark up inline. In [this case](https://github.com/tornadoweb/tornado/blob/b6998247702a960f0c73004a6b5f8ae775607c03/tornado/platform/auto.pyi), a stub file imports and reexports definitions from a python file. With `-s`, python imports are silently ignored.

My suggestion would be to solve this by making `-s` apply only to imports that appear in `.py` files; imports in `.pyi` files would always be processed regardless of their target.

(Versions tested: tornadoweb/tornado@b69982477, python/mypy@ac193cf9dd, python/typeshed@68306484a)



Discussion:

Resolving patch:
diff --git a/mypy/build.py b/mypy/build.py
--- a/mypy/build.py
+++ b/mypy/build.py
@@ -969,12 +969,18 @@ def __init__(self,
                 file_id = '__builtin__'
             path = find_module(file_id, manager.lib_path)
             if path:
-                # In silent mode, don't import .py files.
+                # In silent mode, don't import .py files, except from stubs.
                 if (SILENT_IMPORTS in manager.flags and
                         path.endswith('.py') and (caller_state or is_ancestor)):
-                    path = None
-                    manager.missing_modules.add(id)
-                    raise ModuleNotFound
+                    # (Never silence builtins, even if it's a .py file;
+                    # this can happen in tests!)
+                    if (id != 'builtins' and
+                        not ((caller_state and
+                              caller_state.tree and
+                              caller_state.tree.is_stub))):
+                        path = None
+                        manager.missing_modules.add(id)
+                        raise ModuleNotFound
             else:
                 # Could not find a module.  Typically the reason is a
                 # misspelled module name, missing stub, module not in


Test patch:
diff --git a/mypy/test/data/check-modules.test b/mypy/test/data/check-modules.test
--- a/mypy/test/data/check-modules.test
+++ b/mypy/test/data/check-modules.test
@@ -698,3 +698,21 @@ class C(B):
     pass
 [out]
 tmp/bar.py:1: error: Module has no attribute 'B'
+
+[case testStubImportNonStubWhileSilent]
+# cmd: mypy -m main
+[file main.py]
+# flags: silent-imports
+from stub import x # Permitted
+from other import y # Disallowed
+x + '' # Error here
+y + '' # But not here
+[file stub.pyi]
+from non_stub import x
+[file non_stub.py]
+x = 42
+[file other.py]
+y = 42
+[builtins fixtures/module.py]
+[out]
+tmp/main.py:4: error: Unsupported left operand type for + ("int")
Repository: python/mypy

Issue:
--use-python-path is an attractive nuisance
It seems that many people try to use `--use-python-path` because they see some missing imports (e.g. django, see #1410) and then mypy crashes on some random problem. Maybe we shouldn't have `--use-python-path` at all? @JukkaL, why was it introduced anyway?



Discussion:
UPDATE: I think it was introduced in response to https://github.com/python/mypy/issues/486 (the diff was 6230e3f9680b7e0056593bb9f9ae5bbdbf960a58 by the same user). But I wish it never existed -- people are really better off ignoring imports for which no stubs exist than trying to deal with the errors and crashes from analyzing 3rd party (or stdlib) libraries that haven't been made mypy-clean.

I tend to agree that `--use-python-path` isn't very useful as such. If #638 was implemented (ignoring errors in library modules) and mypy hardly ever crashed, it _might_ be useful, but currently it's unlikely to be helpful. Let's just remove it?


Resolving patch:
diff --git a/mypy/main.py b/mypy/main.py
--- a/mypy/main.py
+++ b/mypy/main.py
@@ -155,7 +155,7 @@ def parse_version(v):
                         help="don't warn if typeshed is out of sync")
     parser.add_argument('--pdb', action='store_true', help="invoke pdb on fatal error")
     parser.add_argument('--use-python-path', action='store_true',
-                        help="search for modules in sys.path of running Python")
+                        help="an anti-pattern")
     parser.add_argument('--stats', action='store_true', help="dump stats")
     parser.add_argument('--inferstats', action='store_true', help="dump type inference stats")
     parser.add_argument('--custom-typing', metavar='MODULE', help="use a custom typing module")
@@ -180,6 +180,14 @@ def parse_version(v):
 
     args = parser.parse_args()
 
+    # --use-python-path is no longer supported; explain why.
+    if args.use_python_path:
+        parser.error("Sorry, --use-python-path is no longer supported.\n"
+                     "If you are trying this because your code depends on a library module,\n"
+                     "you should really investigate how to obtain stubs for that module.\n"
+                     "See https://github.com/python/mypy/issues/1411 for more discussion."
+                     )
+
     # Check for invalid argument combinations.
     code_methods = sum(bool(c) for c in [args.modules, args.command, args.package, args.files])
     if code_methods == 0:


Test patch:
Repository: python/mypy

Issue:
Error message without line number: Signatures of "__rxxx__" of "Class" and "__xxx__" of "Base" are unsafely overlapping
See for example this failed test run: https://travis-ci.org/python/typeshed/builds/112160905

There are several errors of the form

```
stdlib/2and3/fractions.pyi: error: Signatures of "__radd__" of "Fraction" and "__add__" of "Rational" are unsafely overlapping
```

The errors somehow don't have a line number. (I believe the error is correct, I just would like to know the line number.)



Discussion:
Repro:

```
from typing import TypeVar

T = TypeVar('T', Real)

class Real:
    def __add__(self, other): ...

class Fraction(Real):
    def __radd__(self, other: T) -> T: ...
```


Resolving patch:
diff --git a/mypy/treetransform.py b/mypy/treetransform.py
--- a/mypy/treetransform.py
+++ b/mypy/treetransform.py
@@ -129,6 +129,7 @@ def copy_function_attributes(self, new: FuncItem,
         new.max_pos = original.max_pos
         new.is_overload = original.is_overload
         new.is_generator = original.is_generator
+        new.line = original.line
 
     def duplicate_inits(self,
                         inits: List[AssignmentStmt]) -> List[AssignmentStmt]:


Test patch:
diff --git a/mypy/test/data/check-classes.test b/mypy/test/data/check-classes.test
--- a/mypy/test/data/check-classes.test
+++ b/mypy/test/data/check-classes.test
@@ -1147,6 +1147,17 @@ class X:
 main: note: In member "__radd__" of class "B":
 main:6: error: Signatures of "__radd__" of "B" and "__add__" of "X" are unsafely overlapping
 
+[case testUnsafeOverlappingWithLineNo]
+from typing import TypeVar
+T = TypeVar('T', Real)
+class Real:
+    def __add__(self, other): ...
+class Fraction(Real):
+    def __radd__(self, other: T) -> T: ...
+[out]
+main: note: In member "__radd__" of class "Fraction":
+main:6: error: Signatures of "__radd__" of "Fraction" and "__add__" of "Real" are unsafely overlapping
+
 [case testOverlappingNormalAndInplaceOperatorMethod]
 import typing
 class A:
Repository: python/mypy

Issue:
error with double tuple assignment
Not sure how to describe this with words so apologies if this has already been noted.

mypy has no problem checking `a = b = 1` or `a, b = 1, 2`, but chokes on

```
c = a, b = 1, 2
```

with `KeyError: <mypy.nodes.TupleExpr object at 0x1017950b8>`. Full traceback is [here](https://gist.github.com/d10genes/4427eb29e68521a9b078).



Discussion:
Thanks for reporting this! This is clearly a bug in how mypy handles assignment statements.

The current workaround is to split the assignment into two or more assignment statements, for example:

```
c = 1, 2
a, b = c
```


Resolving patch:
diff --git a/mypy/checker.py b/mypy/checker.py
--- a/mypy/checker.py
+++ b/mypy/checker.py
@@ -1143,6 +1143,8 @@ def visit_assignment_stmt(self, s: AssignmentStmt) -> Type:
         if len(s.lvalues) > 1:
             # Chained assignment (e.g. x = y = ...).
             # Make sure that rvalue type will not be reinferred.
+            if s.rvalue not in self.type_map:
+                self.accept(s.rvalue)
             rvalue = self.temp_node(self.type_map[s.rvalue], s)
             for lv in s.lvalues[:-1]:
                 self.check_assignment(lv, rvalue, s.type is None)


Test patch:
diff --git a/mypy/test/data/check-tuples.test b/mypy/test/data/check-tuples.test
--- a/mypy/test/data/check-tuples.test
+++ b/mypy/test/data/check-tuples.test
@@ -364,6 +364,12 @@ b = ''
 a = '' # E: Incompatible types in assignment (expression has type "str", variable has type "int")
 b = 1  # E: Incompatible types in assignment (expression has type "int", variable has type "str")
 
+[case testMultipleAssignmentWithMixedVariables]
+a = b, c = 1, 1
+x, y = p, q = 1, 1
+u, v, w = r, s = 1, 1 # E: Need more than 2 values to unpack (3 expected)
+d, e = f, g, h = 1, 1 # E: Need more than 2 values to unpack (3 expected)
+
 
 -- Assignment to starred expressions
 -- ---------------------------------
Repository: python/mypy

Issue:
Incorrect analysis of constant conditions in elif blocks.
This line:
https://github.com/python/mypy/blob/master/mypy/semanal.py#L2647
repeatedly marks the final reachable block as unreachable. 
It should mark all the _following_ blocks as unreachable.



Discussion:

Resolving patch:
diff --git a/mypy/semanal.py b/mypy/semanal.py
--- a/mypy/semanal.py
+++ b/mypy/semanal.py
@@ -2644,7 +2644,7 @@ def infer_reachability_of_if_statement(s: IfStmt,
             # This condition is always true, so all of the remaining
             # elif/else bodies will never be executed.
             for body in s.body[i + 1:]:
-                mark_block_unreachable(s.body[i])
+                mark_block_unreachable(body)
             if s.else_body:
                 mark_block_unreachable(s.else_body)
             break


Test patch:
diff --git a/mypy/test/data/check-unreachable-code.test b/mypy/test/data/check-unreachable-code.test
--- a/mypy/test/data/check-unreachable-code.test
+++ b/mypy/test/data/check-unreachable-code.test
@@ -98,3 +98,16 @@ if PY3:
 else:
     class X(object):
         pass
+
+[case testUnreachabilityAndElifPY3]
+def f(): pass
+PY3 = f()
+if PY3:
+    pass
+elif 1:
+    import nonexistent
+    1 + ''
+else:
+    import bad_name
+    1 + ''
+[out]
Repository: python/mypy

Issue:
There's a difference between TypeVar('T', bound=Callable) and TypeVar('T', bound=Callable[..., Any])
@rwbarton Any idea? I found this in our internal code base, haven't come up with a small repro yet. Plain `Callable` is supposed to mean the same as `Callable[..., Any]` but in this case it's apparently not the case. I had some code that gave a long list of errors when I used the shorter form. OTOH I had some other code where it worked...



Discussion:
I can reproduce this by making that change to test `testBoundOnDecorator`. Will investigate.


Resolving patch:
diff --git a/mypy/typeanal.py b/mypy/typeanal.py
--- a/mypy/typeanal.py
+++ b/mypy/typeanal.py
@@ -210,12 +210,13 @@ def visit_ellipsis_type(self, t: EllipsisType) -> Type:
     def analyze_callable_type(self, t: UnboundType) -> Type:
         fallback = self.builtin_type('builtins.function')
         if len(t.args) == 0:
-            # Callable (bare)
+            # Callable (bare). Treat as Callable[..., Any].
             return CallableType([AnyType(), AnyType()],
                                 [nodes.ARG_STAR, nodes.ARG_STAR2],
                                 [None, None],
                                 ret_type=AnyType(),
-                                fallback=fallback)
+                                fallback=fallback,
+                                is_ellipsis_args=True)
         elif len(t.args) == 2:
             ret_type = t.args[1].accept(self)
             if isinstance(t.args[0], TypeList):


Test patch:
diff --git a/mypy/test/data/check-functions.test b/mypy/test/data/check-functions.test
--- a/mypy/test/data/check-functions.test
+++ b/mypy/test/data/check-functions.test
@@ -1531,6 +1531,11 @@ from typing import Callable, Any
 
 def foo(f: Callable) -> bool:
     return f()
+
+def f1() -> bool:
+    return False
+
+foo(f1)
 [builtins fixtures/bool.py]
 
 [case testFunctionNestedWithinWith]
Repository: python/mypy

Issue:
error: Cannot infer type of lambda
Repro:

``` python
from typing import TypeVar, Callable
T = TypeVar('T')
def foo(arg: Callable[..., T]) -> None: pass
foo(lambda: 1)  # E: Cannot infer type of lambda
x = lambda: 1
foo(x)  # OK
```

That error message was introduced in 6edd1b96f39189877b7650082dc0e2eec7b4b34c ("Fix lambda type inference special cases").



Discussion:
Even the generic function is unneeded here.

``` py
from typing import Callable
y = lambda: 1  # type: Callable[..., int]  # E: Cannot infer type of lambda
```


Resolving patch:
diff --git a/mypy/checkexpr.py b/mypy/checkexpr.py
--- a/mypy/checkexpr.py
+++ b/mypy/checkexpr.py
@@ -1299,6 +1299,14 @@ def infer_lambda_type_using_context(self, e: FuncExpr) -> CallableType:
 
         arg_kinds = [arg.kind for arg in e.arguments]
 
+        if callable_ctx.is_ellipsis_args:
+            # Fill in Any arguments to match the arguments of the lambda.
+            callable_ctx = callable_ctx.copy_modified(
+                is_ellipsis_args=False,
+                arg_types=[AnyType()] * len(arg_kinds),
+                arg_kinds=arg_kinds
+            )
+
         if callable_ctx.arg_kinds != arg_kinds:
             # Incompatible context; cannot use it to infer types.
             self.chk.fail(messages.CANNOT_INFER_LAMBDA_TYPE, e)


Test patch:
diff --git a/mypy/test/data/check-inference-context.test b/mypy/test/data/check-inference-context.test
--- a/mypy/test/data/check-inference-context.test
+++ b/mypy/test/data/check-inference-context.test
@@ -593,6 +593,24 @@ main:2: error: Cannot infer type of lambda
 main:3: error: Cannot infer type of lambda
 main:3: error: Incompatible types in assignment (expression has type Callable[[], A], variable has type Callable[[A], A])
 
+[case testEllipsisContextForLambda]
+from typing import Callable
+f1 = lambda x: 1 # type: Callable[..., int]
+f2 = lambda: 1 # type: Callable[..., int]
+f3 = lambda *args, **kwargs: 1 # type: Callable[..., int]
+f4 = lambda x: x # type: Callable[..., int]
+g = lambda x: 1 # type: Callable[..., str]
+[builtins fixtures/dict.py]
+[out]
+main:6: error: Incompatible return value type: expected builtins.str, got builtins.int
+main:6: error: Incompatible types in assignment (expression has type Callable[[Any], int], variable has type Callable[..., str])
+
+[case testEllipsisContextForLambda2]
+from typing import TypeVar, Callable
+T = TypeVar('T')
+def foo(arg: Callable[..., T]) -> None: pass
+foo(lambda: 1)
+
 
 -- Overloads + generic functions
 -- -----------------------------