internlm/SWE-Fixer-Train-110K huggingface dataset:
Repository: python/mypy

Issue:
Don't run PEP561 tests in runtests.py by default
Now that PEP561 tests run sequentially (#12857), they can take a big fraction of the
entire runtests.py runtime. I think that we should make the tests opt-in, similar to some 
of the slow mypyc tests. They will run in CI always in any case.

Anybody want to help with this?
Don't run PEP561 tests in runtests.py by default
Now that PEP561 tests run sequentially (#12857), they can take a big fraction of the
entire runtests.py runtime. I think that we should make the tests opt-in, similar to some 
of the slow mypyc tests. They will run in CI always in any case.

Anybody want to help with this?


Discussion:

Resolving patch:
diff --git a/runtests.py b/runtests.py
--- a/runtests.py
+++ b/runtests.py
@@ -36,9 +36,14 @@
 ]
 
 
+# This must be enabled by explicitly including 'pytest-extra' on the command line
+PYTEST_OPT_IN = [PEP561]
+
+
 # These must be enabled by explicitly including 'mypyc-extra' on the command line.
 MYPYC_OPT_IN = [MYPYC_RUN, MYPYC_RUN_MULTI]
 
+
 # We split the pytest run into three parts to improve test
 # parallelization. Each run should have tests that each take a roughly similar
 # time to run.
@@ -64,14 +69,14 @@
          ERROR_STREAM])],
 
     # Test cases that might take minutes to run
-    'pytest-slower': ['pytest', '-q', '-k', ' or '.join(
-        [PEP561])],
+    'pytest-extra': ['pytest', '-q', '-k', ' or '.join(PYTEST_OPT_IN)],
 
     # Test cases to run in typeshed CI
     'typeshed-ci': ['pytest', '-q', '-k', ' or '.join([CMDLINE,
                                                        EVALUATION,
                                                        SAMPLES,
                                                        TYPESHED])],
+
     # Mypyc tests that aren't run by default, since they are slow and rarely
     # fail for commits that don't touch mypyc
     'mypyc-extra': ['pytest', '-q', '-k', ' or '.join(MYPYC_OPT_IN)],
@@ -80,7 +85,8 @@
 # Stop run immediately if these commands fail
 FAST_FAIL = ['self', 'lint']
 
-DEFAULT_COMMANDS = [cmd for cmd in cmds if cmd not in ('mypyc-extra', 'typeshed-ci')]
+EXTRA_COMMANDS = ('pytest-extra', 'mypyc-extra', 'typeshed-ci')
+DEFAULT_COMMANDS = [cmd for cmd in cmds if cmd not in EXTRA_COMMANDS]
 
 assert all(cmd in cmds for cmd in FAST_FAIL)
 
@@ -124,7 +130,8 @@ def main() -> None:
     if not set(args).issubset(cmds):
         print("usage:", prog, " ".join('[%s]' % k for k in cmds))
         print()
-        print('Run the given tests. If given no arguments, run everything except mypyc-extra.')
+        print('Run the given tests. If given no arguments, run everything except'
+              + ' pytest-extra and mypyc-extra.')
         exit(1)
 
     if not args:


Test patch:
Repository: python/mypy

Issue:
Revert LteralString support from vendored typeshed copy
We don't properly support `LteralString` anyway, so its use in typeshed causes some false positives. We need to locally revert https://github.com/python/typeshed/pull/7725 for now.


Discussion:

Resolving patch:
diff --git a/mypy/config_parser.py b/mypy/config_parser.py
--- a/mypy/config_parser.py
+++ b/mypy/config_parser.py
@@ -11,8 +11,8 @@
 else:
     import tomli as tomllib
 
-from typing import (Any, Callable, Dict, List, Mapping, MutableMapping,  Optional, Sequence,
-                    TextIO, Tuple, Union)
+from typing import (Any, Callable, Dict, List, Mapping, MutableMapping, Optional, Sequence,
+                    TextIO, Tuple, Union, Iterable)
 from typing_extensions import Final, TypeAlias as _TypeAlias
 
 from mypy import defaults
@@ -179,7 +179,8 @@ def parse_config_file(options: Options, set_strict_flags: Callable[[], None],
     if filename is not None:
         config_files: Tuple[str, ...] = (filename,)
     else:
-        config_files = tuple(map(os.path.expanduser, defaults.CONFIG_FILES))
+        config_files_iter: Iterable[str] = map(os.path.expanduser, defaults.CONFIG_FILES)
+        config_files = tuple(config_files_iter)
 
     config_parser = configparser.RawConfigParser()
 


Test patch:
Repository: python/mypy

Issue:
(🎁) Warn when `NoneType` is used in a type
`NoneType` is a mangled up mess in the type realm and should never be used in a typing context.

With the special case of `None` being a secret alias to `NoneType` I think that many, many people are going to try to use `NoneType` as an annotation. Adding a warning for the 99% of the time that it's invalidly used would be a great one.

I understand that there are always edge cases, and disabling those warnings with a type comment would be the way to go IMO. 

```py
from types import NoneType

def foo(a: NoneType):  # sus alert, NoneType is a mess in the type realm, you should use `None` or `type[None]` instead.
    print(isinstance(a, NoneType))
```


Discussion:
Some more context from the duplicate issue #13139 below.

Hints:
* Fix this in `mypy.typeanal`
* Currently `types.NoneType` is treated as an `mypy.types.Instance` type but `None` is mapped to `mypy.types.NoneType`
* If we'd create an `Instance` with `fullname` equal to `types.NoneType`, generate an error

Example illustrating the problem:
```
from types import NoneType

def f(x: NoneType) -> None:
    pass

f(None)   # Incompatible type None, expected NoneType
```
@JukkaL I have partially improved this case in basedmypy.

basedmypy:
```py
from types import NoneType

def foo(n: NoneType):  # error: Variable "types.NoneType" is not valid as a type
    print("hi")

nt: type[None] = NoneType  # no error
```

just by simply making `NoneType` an alias instead of a class def. It's not perfect, but I think it's a big improvement.

https://github.com/KotlinIsland/basedmypy/commit/f424a34b4fe774d920a744ca2fc981264958a0b5

Resolving patch:
diff --git a/mypy/typeanal.py b/mypy/typeanal.py
--- a/mypy/typeanal.py
+++ b/mypy/typeanal.py
@@ -508,6 +508,11 @@ def analyze_type_with_type_info(
             # Create a named TypedDictType
             return td.copy_modified(item_types=self.anal_array(list(td.items.values())),
                                     fallback=instance)
+
+        if info.fullname == 'types.NoneType':
+            self.fail("NoneType should not be used as a type, please use None instead", ctx)
+            return NoneType(ctx.line, ctx.column)
+
         return instance
 
     def analyze_unbound_type_without_type_info(self, t: UnboundType, sym: SymbolTableNode,


Test patch:
diff --git a/test-data/unit/check-python310.test b/test-data/unit/check-python310.test
--- a/test-data/unit/check-python310.test
+++ b/test-data/unit/check-python310.test
@@ -1591,3 +1591,12 @@ def test_func(test_str: str) -> str:
             return "special"
         case _:
             return "other"
+
+
+[case testNoneTypeWarning]
+from types import NoneType
+
+def foo(x: NoneType): # E: NoneType should not be used as a type, please use None instead
+    reveal_type(x) # N: Revealed type is "None"
+
+[builtins fixtures/tuple.pyi]
Repository: python/mypy

Issue:
nonlocal in class def reports an error.
**Bug Report**

A class def containing a nonlocal declaration is flagged as an error, even though this is legal Python.

**To Reproduce**

```
X.py:

def f():
    a = 0
    class C:
        nonlocal a
        a = 1
```
mypy output:
`error: nonlocal declaration not allowed at module level`

**Fix**
Change one line in `SemanticAnalyzer.visit_nonlocal_decl`
from:
```
	def visit_nonlocal_decl(self, d: NonlocalDecl) -> None:
		self.statement = d
!		if not self.is_func_scope():
			self.fail("nonlocal declaration not allowed at module level", d)
		...
```
to:
```
	def visit_nonlocal_decl(self, d: NonlocalDecl) -> None:
		self.statement = d
!		if self.is_module_scope():
			self.fail("nonlocal declaration not allowed at module level", d)
		...
```
The error message now disappears.  Note, it still appears at the module level, as it is a SyntaxError in Python.

**Your Environment**

- Mypy version used: 0.931



Discussion:
I'm not able to repro this error with the latest version of mypy. It looks like you're using mypy 0.931 which is quite old. Have you tried upgrading to the latest?
Thanks for the report! (And I can repro, I suspect Eric might be getting tripped up by the fact that `--check-untyped-defs` (or `--strict`) are not the default).

If you open a PR, I'll merge.
Yep, good point. When I specify `--check-untyped-defs`, I see the error too.

Resolving patch:
diff --git a/mypy/semanal.py b/mypy/semanal.py
--- a/mypy/semanal.py
+++ b/mypy/semanal.py
@@ -3807,7 +3807,7 @@ def visit_global_decl(self, g: GlobalDecl) -> None:
 
     def visit_nonlocal_decl(self, d: NonlocalDecl) -> None:
         self.statement = d
-        if not self.is_func_scope():
+        if self.is_module_scope():
             self.fail("nonlocal declaration not allowed at module level", d)
         else:
             for name in d.names:


Test patch:
diff --git a/test-data/unit/semanal-basic.test b/test-data/unit/semanal-basic.test
--- a/test-data/unit/semanal-basic.test
+++ b/test-data/unit/semanal-basic.test
@@ -382,6 +382,29 @@ MypyFile:1(
           ExpressionStmt:6(
             NameExpr(x [l])))))))
 
+[case testNonlocalClass]
+def f() -> None:
+    a = 0
+    class C:
+        nonlocal a
+        a = 1
+[out]
+MypyFile:1(
+  FuncDef:1(
+    f
+    def ()
+    Block:1(
+      AssignmentStmt:2(
+        NameExpr(a* [l])
+        IntExpr(0))
+      ClassDef:3(
+        C
+        NonlocalDecl:4(
+          a)
+        AssignmentStmt:5(
+          NameExpr(a* [m])
+          IntExpr(1))))))
+
 [case testMultipleNamesInNonlocalDecl]
 def g():
     x, y = None, None
Repository: python/mypy

Issue:
sys.path change breaks editable installs of cwd
The new sys.path logic introduced in https://github.com/python/mypy/pull/11143 carefully avoids putting cwd onto sys.path, which is probably generally a good thing. However, the logic incorrectly removes cwd from sys.path in cases where the current directory has been editably installed (i.e. is on sys.path for reasons unrelated to being the current directory).

I'm not the most familiar with the code, but my understanding is this often doesn't really matter, since cwd should be on `search_paths.python_path`. However, it looks like we only look for stub packages on `search_paths.package_path` (which is where cwd is now removed from).

My concern here isn't theoretical. It came up in #12931 where when trying to reproduce, stubtest was unable to find the django stubs.

Depending on how serious we feel this concern is / how strictly backward compatible we want to be, there are several options for working around this. Curious for your opinions!

cc @AWhetter @ethanhs 


Discussion:
I think the best thing to do is search for stub packages in the python path as well. We've had requests for this in the past, and with https://github.com/python/mypy/issues/8545 any extra packages can be disabled if need be.

(Also from a consistency standpoint this would be simpler)

Resolving patch:
diff --git a/mypy/pyinfo.py b/mypy/pyinfo.py
--- a/mypy/pyinfo.py
+++ b/mypy/pyinfo.py
@@ -10,9 +10,6 @@
 import sys
 import sysconfig
 
-if __name__ == '__main__':
-    sys.path = sys.path[1:]  # we don't want to pick up mypy.types
-
 MYPY = False
 if MYPY:
     from typing import List
@@ -29,10 +26,17 @@ def getsearchdirs():
     )
     stdlib = sysconfig.get_path("stdlib")
     stdlib_ext = os.path.join(stdlib, "lib-dynload")
-    cwd = os.path.abspath(os.getcwd())
-    excludes = set([cwd, stdlib_zip, stdlib, stdlib_ext])
-
-    abs_sys_path = (os.path.abspath(p) for p in sys.path)
+    excludes = set([stdlib_zip, stdlib, stdlib_ext])
+
+    # Drop the first entry of sys.path
+    # - If pyinfo.py is executed as a script (in a subprocess), this is the directory
+    #   containing pyinfo.py
+    # - Otherwise, if mypy launched via console script, this is the directory of the script
+    # - Otherwise, if mypy launched via python -m mypy, this is the current directory
+    # In all cases, this is safe to drop
+    # Note that mypy adds the cwd to SearchPaths.python_path, so we still find things on the
+    # cwd consistently (the return value here sets SearchPaths.package_path)
+    abs_sys_path = (os.path.abspath(p) for p in sys.path[1:])
     return [p for p in abs_sys_path if p not in excludes]
 
 


Test patch:
Repository: python/mypy

Issue:
PEP 561 test failures due to pyinfo break wheel builds on macOS
Traceback from https://github.com/mypyc/mypy_mypyc-wheels/runs/7376619798?check_suite_focus=true (also reproduces locally on macOS):
```
  ______________________________ testTypedPkgSimple ______________________________
  [gw1] darwin -- Python 3.10.0 /private/var/folders/24/8k48jl6d249_n_qfxwsl6xvm0000gn/T/tmpl3v_i2pp/bin/python
  data: /Users/runner/work/mypy_mypyc-wheels/mypy_mypyc-wheels/mypy/test-data/unit/pep561.test:14:
  ../tmpl3v_i2pp/lib/python3.10/site-packages/mypy/test/testpep561.py:31: in run_case
      test_pep561(test_case)
  ../tmpl3v_i2pp/lib/python3.10/site-packages/mypy/test/testpep561.py:142: in test_pep561
      out, err, returncode = mypy.api.run(cmd_line)
  mypy/api.py:69: in run
      ???
  mypy/api.py:58: in _run
      ???
  mypy/api.py:69: in <lambda>
      ???
  mypy/main.py:96: in main
      ???
  mypy/main.py:173: in run_build
      ???
  mypy/build.py:154: in build
      ???
  mypy/build.py:186: in _build
      ???
  mypy/modulefinder.py:844: in compute_search_paths
      ???
  mypy/modulefinder.py:748: in get_search_dirs
      ???
  /Library/Frameworks/Python.framework/Versions/3.10/lib/python3.10/subprocess.py:420: in check_output
      return run(*popenargs, stdout=PIPE, timeout=timeout, check=True,
  /Library/Frameworks/Python.framework/Versions/3.10/lib/python3.10/subprocess.py:524: in run
      raise CalledProcessError(retcode, process.args,
  E   subprocess.CalledProcessError: Command '['/var/folders/24/8k48jl6d249_n_qfxwsl6xvm0000gn/T/tmp1l0jvq71/bin/python', '/private/var/folders/24/8k48jl6d249_n_qfxwsl6xvm0000gn/T/tmpl3v_i2pp/lib/python3.10/site-packages/mypy/pyinfo.py', 'getsearchdirs']' returned non-zero exit status 1.
```

I have checked that this is a side effect of #13161.


Discussion:

Resolving patch:
diff --git a/mypy/pyinfo.py b/mypy/pyinfo.py
--- a/mypy/pyinfo.py
+++ b/mypy/pyinfo.py
@@ -14,6 +14,15 @@
 if MYPY:
     from typing import List
 
+if __name__ == '__main__':
+    # HACK: We don't want to pick up mypy.types as the top-level types
+    #       module. This could happen if this file is run as a script.
+    #       This workaround fixes it.
+    old_sys_path = sys.path
+    sys.path = sys.path[1:]
+    import types  # noqa
+    sys.path = old_sys_path
+
 
 def getsearchdirs():
     # type: () -> List[str]


Test patch:
Repository: python/mypy

Issue:
stubtest false positive for Protocol `__init__`
From `python -m mypy.stubtest --custom-typeshed-dir . wsgiref`:

```
error: wsgiref.types._Readable.__init__ is inconsistent, stub does not have *args argument "args"
Stub: at line 87
def (self: builtins.object)
Runtime: at line 1912 in file /Users/jelle/py/cpython/Lib/typing.py
def (self, *args, **kwargs)
```

The stub and implementation are exactly the same.

Stubtest picks up the `__init__` method the runtime injects for Protocols, but it's not useful.


Discussion:

Resolving patch:
diff --git a/mypy/stubtest.py b/mypy/stubtest.py
--- a/mypy/stubtest.py
+++ b/mypy/stubtest.py
@@ -344,6 +344,14 @@ class SubClass(runtime):  # type: ignore
         for m in cast(Any, vars)(runtime)
         if not is_probably_private(m) and m not in IGNORABLE_CLASS_DUNDERS
     )
+    # Special-case the __init__ method for Protocols
+    #
+    # TODO: On Python <3.11, __init__ methods on Protocol classes
+    # are silently discarded and replaced.
+    # However, this is not the case on Python 3.11+.
+    # Ideally, we'd figure out a good way of validating Protocol __init__ methods on 3.11+.
+    if stub.is_protocol:
+        to_check.discard("__init__")
 
     for entry in sorted(to_check):
         mangled_entry = entry
@@ -1090,6 +1098,7 @@ def verify_typealias(
     {
         # Special attributes
         "__dict__",
+        "__annotations__",
         "__text_signature__",
         "__weakref__",
         "__del__",  # Only ever called when an object is being deleted, who cares?


Test patch:
diff --git a/mypy/test/teststubtest.py b/mypy/test/teststubtest.py
--- a/mypy/test/teststubtest.py
+++ b/mypy/test/teststubtest.py
@@ -1033,16 +1033,17 @@ def test_protocol(self) -> Iterator[Case]:
             from typing_extensions import Protocol
 
             class X(Protocol):
+                bar: int
                 def foo(self, x: int, y: bytes = ...) -> str: ...
             """,
             runtime="""
             from typing_extensions import Protocol
 
             class X(Protocol):
+                bar: int
                 def foo(self, x: int, y: bytes = ...) -> str: ...
             """,
-            # TODO: this should not be an error, #12820
-            error="X.__init__"
+            error=None
         )
 
     @collect_cases
Repository: python/mypy

Issue:
Cannot access "__init__" directly
Calling `__init__` explicitly is a valid (albeit uncommon) pattern, for example after calling explicitly `__new__`; however mypy considers it an error:

```py
from __future__ import annotations


class Foo:
    def __init__(self, a: int):
        self.a = a


class Bar(Foo):
    def __new__(cls, a: int) -> Bar:
        self = super().__new__(cls)
        # ....
        self.__init__(a)
        # ....
        return self


if __name__ == '__main__':
    print(Bar(1))
```


Discussion:
Interestingly, this check goes back to 2013 (and beyond, in Alore). See the reasoning here: https://github.com/python/mypy/issues/89
The error message isn't great, so I would accept a PR improving it
I'm not sure I'm following the reasoning in the mentioned ticket but from a quick [code search](https://sourcegraph.com/search?q=context:global+%5Cw%5C.__init__%5C%28%5B%5Es%5Cs%5D%5Ba-z%5D+lang:python+&patternType=regexp) there seem to be quite a few cases of calling `__init__` directly, a lot of them to initialize an instance created with `__new__`.

> As an exception, it is valid to use super() to access `__init__` (we know the exact method to call).

Even this exception is limited:
```py
    def __init__(self, a: int):
        # this typechecks
        super().__init__(a)

        # this doesn't
        s = super()
        s.__init__(a)
```


Resolving patch:
diff --git a/mypy/message_registry.py b/mypy/message_registry.py
--- a/mypy/message_registry.py
+++ b/mypy/message_registry.py
@@ -74,7 +74,10 @@ def format(self, *args: object, **kwargs: object) -> "ErrorMessage":
 TUPLE_INDEX_OUT_OF_RANGE: Final = ErrorMessage("Tuple index out of range")
 INVALID_SLICE_INDEX: Final = ErrorMessage("Slice index must be an integer or None")
 CANNOT_INFER_LAMBDA_TYPE: Final = ErrorMessage("Cannot infer type of lambda")
-CANNOT_ACCESS_INIT: Final = 'Cannot access "__init__" directly'
+CANNOT_ACCESS_INIT: Final = (
+    'Accessing "__init__" on an instance is unsound, since instance.__init__ could be from'
+    ' an incompatible subclass'
+)
 NON_INSTANCE_NEW_TYPE: Final = ErrorMessage('"__new__" must return a class instance (got {})')
 INVALID_NEW_TYPE: Final = ErrorMessage('Incompatible return type for "__new__"')
 BAD_CONSTRUCTOR_TYPE: Final = ErrorMessage("Unsupported decorated constructor type")


Test patch:
diff --git a/test-data/unit/check-classes.test b/test-data/unit/check-classes.test
--- a/test-data/unit/check-classes.test
+++ b/test-data/unit/check-classes.test
@@ -840,7 +840,7 @@ from typing import Any, cast
 class A:
     def __init__(self, a: 'A') -> None: pass
 a = None # type: A
-a.__init__(a)  # E: Cannot access "__init__" directly
+a.__init__(a)  # E: Accessing "__init__" on an instance is unsound, since instance.__init__ could be from an incompatible subclass
 (cast(Any, a)).__init__(a)
 
 [case testDeepInheritanceHierarchy]
Repository: python/mypy

Issue:
`None` incompatible with `SupportsStr`
**Bug Report**

I have a protocol, `SupportsStr`:
```py
class SupportsStr(Protocol):
    def __str__(self) -> str:
        ...
```

`None` is, surprisingly, not compliant with it.

**To Reproduce**

```py
from typing import Protocol

class SupportsStr(Protocol):
    def __str__(self) -> str:
        ...
        
x: SupportsStr = None
```

https://mypy-play.net/?mypy=latest&python=3.10&gist=3b07f4f4125f4fdeff96050cd7309494

**Expected Behavior**

I expect `None` to be compatible with `SupportsStr`.

**Actual Behavior**

In reality, `None` is not compatible with `SupportsStr`.

**Your Environment**

- Mypy version used: 0.961
- Python version used: 3.10



Discussion:

Resolving patch:
diff --git a/mypy/subtypes.py b/mypy/subtypes.py
--- a/mypy/subtypes.py
+++ b/mypy/subtypes.py
@@ -239,7 +239,8 @@ def visit_none_type(self, left: NoneType) -> bool:
                 members = self.right.type.protocol_members
                 # None is compatible with Hashable (and other similar protocols). This is
                 # slightly sloppy since we don't check the signature of "__hash__".
-                return not members or members == ["__hash__"]
+                # None is also compatible with `SupportsStr` protocol.
+                return not members or all(member in ("__hash__", "__str__") for member in members)
             return False
         else:
             return True


Test patch:
diff --git a/test-data/unit/check-protocols.test b/test-data/unit/check-protocols.test
--- a/test-data/unit/check-protocols.test
+++ b/test-data/unit/check-protocols.test
@@ -2731,6 +2731,20 @@ class EmptyProto(Protocol): ...
 
 def hh(h: EmptyProto) -> None: pass
 hh(None)
+
+# See https://github.com/python/mypy/issues/13081
+class SupportsStr(Protocol):
+    def __str__(self) -> str: ...
+
+def ss(s: SupportsStr) -> None: pass
+ss(None)
+
+class HashableStr(Protocol):
+    def __str__(self) -> str: ...
+    def __hash__(self) -> int: ...
+
+def hs(n: HashableStr) -> None: pass
+hs(None)
 [builtins fixtures/tuple.pyi]