Repository: python/mypy

Issue:
Objects matching `SupportsKeysAndGetItem[str, Any]` cannot be used after `**`
**Bug Report**

Given the following code:

```python
from __future__ import annotations

from collections.abc import Iterable
from typing import Any


class MyThing:
    def keys(self) -> Iterable[str]:
        return ['foo']

    def __getitem__(self, __key: str) -> Any:
        return 'bar'


def foo(**args: Any) -> None:
    print(args)


foo(**MyThing())
```

`mypy` generates an error at line 19: `Argument after ** must be a mapping, not "MyThing"`

**Expected Behavior**

No error should be generated since it works at runtime. This has been [discussed](https://github.com/python/typing/discussions/1375) over in the Python Type School, and `pyright` [has been updated](https://github.com/microsoft/pyright/issues/4826) to support this.

**Your Environment**

- Mypy version used: 1.1.1
- Mypy command-line flags: n/a
- Mypy configuration options from `mypy.ini` (and other config files): n/a
- Python version used: 3.10.10

<!-- You can freely edit this text, please remove all the lines you believe are unnecessary. -->



Discussion:
Should be https://github.com/python/mypy/blob/c99133f405f286ed3429c809e9ae2cb3faaa2ceb/mypy/checkexpr.py#L2129 if someone wants to pick this up!
@hauntsaninja so that could simply be changed to `self.chk.named_type("_typeshed.SupportsKeysAndGetItem")`?
There's a good chance the core of that change is that simple. Will need some effort in unit tests as well if you do it that way, since I don't think we currently have a `_typeshed` stub in unit tests.
@hauntsaninja Well, I gave it a shot. I figured out that there are a couple of other places that need to be changed:

https://github.com/python/mypy/blob/89469ac51aff1eefcd693d87d621577370ddedb4/mypy/checkexpr.py#L4933-L4950

Both of those need to be changed from `typing.Mapping` to `_typeshed.SupportsKeysAndGetItem`. With the change to the line you noted as well as these extra changes, it seems that it fixes the issue (when I run tests by calling `python -m mypy foo.py` in my venv).

However, I'm unable to get the unit tests to pick up a `_typeshed.pyi` file in `test-data/unit/lib-stub` or in `test-data/unit/fixtures` (I had to modify `mypy/test/data.py` to attempt the latter, but it still didn't work). It's been a while since I've worked on mypy, so I may be missing something really simple. The changed calls always raise `KeyError: '_typeshed'` in the unit test runner.
Might need to do something like `import _typeshed` in the `builtins.pyi` fixture you're using so that it's always part of the mypy build. Also feel free to open a draft PR and I can help fix up whenever I get time. The test fixtures are the most annoying part of mypy dev
(edit: maybe best to do this in a custom builtins.pyi fixture)

Resolving patch:
diff --git a/mypy/checkexpr.py b/mypy/checkexpr.py
--- a/mypy/checkexpr.py
+++ b/mypy/checkexpr.py
@@ -2126,7 +2126,9 @@ def check_argument_types(
                 if actual_kind == nodes.ARG_STAR2 and not self.is_valid_keyword_var_arg(
                     actual_type
                 ):
-                    is_mapping = is_subtype(actual_type, self.chk.named_type("typing.Mapping"))
+                    is_mapping = is_subtype(
+                        actual_type, self.chk.named_type("_typeshed.SupportsKeysAndGetItem")
+                    )
                     self.msg.invalid_keyword_var_arg(actual_type, is_mapping, context)
                 expanded_actual = mapper.expand_actual_type(
                     actual_type, actual_kind, callee.arg_names[i], callee_arg_kind
@@ -4346,7 +4348,11 @@ def visit_dict_expr(self, e: DictExpr) -> Type:
             for arg in stargs:
                 if rv is None:
                     constructor = CallableType(
-                        [self.chk.named_generic_type("typing.Mapping", [kt, vt])],
+                        [
+                            self.chk.named_generic_type(
+                                "_typeshed.SupportsKeysAndGetItem", [kt, vt]
+                            )
+                        ],
                         [nodes.ARG_POS],
                         [None],
                         self.chk.named_generic_type("builtins.dict", [kt, vt]),
@@ -4936,14 +4942,14 @@ def is_valid_keyword_var_arg(self, typ: Type) -> bool:
             is_subtype(
                 typ,
                 self.chk.named_generic_type(
-                    "typing.Mapping",
+                    "_typeshed.SupportsKeysAndGetItem",
                     [self.named_type("builtins.str"), AnyType(TypeOfAny.special_form)],
                 ),
             )
             or is_subtype(
                 typ,
                 self.chk.named_generic_type(
-                    "typing.Mapping", [UninhabitedType(), UninhabitedType()]
+                    "_typeshed.SupportsKeysAndGetItem", [UninhabitedType(), UninhabitedType()]
                 ),
             )
             or isinstance(typ, ParamSpecType)
diff --git a/mypy/checkstrformat.py b/mypy/checkstrformat.py
--- a/mypy/checkstrformat.py
+++ b/mypy/checkstrformat.py
@@ -844,10 +844,14 @@ def build_dict_type(self, expr: FormatStringExpr) -> Type:
         any_type = AnyType(TypeOfAny.special_form)
         if isinstance(expr, BytesExpr):
             bytes_type = self.chk.named_generic_type("builtins.bytes", [])
-            return self.chk.named_generic_type("typing.Mapping", [bytes_type, any_type])
+            return self.chk.named_generic_type(
+                "_typeshed.SupportsKeysAndGetItem", [bytes_type, any_type]
+            )
         elif isinstance(expr, StrExpr):
             str_type = self.chk.named_generic_type("builtins.str", [])
-            return self.chk.named_generic_type("typing.Mapping", [str_type, any_type])
+            return self.chk.named_generic_type(
+                "_typeshed.SupportsKeysAndGetItem", [str_type, any_type]
+            )
         else:
             assert False, "Unreachable"
 
diff --git a/mypyc/test-data/fixtures/ir.py b/mypyc/test-data/fixtures/ir.py
--- a/mypyc/test-data/fixtures/ir.py
+++ b/mypyc/test-data/fixtures/ir.py
@@ -1,6 +1,7 @@
 # These builtins stubs are used implicitly in AST to IR generation
 # test cases.
 
+import _typeshed
 from typing import (
     TypeVar, Generic, List, Iterator, Iterable, Dict, Optional, Tuple, Any, Set,
     overload, Mapping, Union, Callable, Sequence, FrozenSet, Protocol


Test patch:
diff --git a/mypy/test/data.py b/mypy/test/data.py
--- a/mypy/test/data.py
+++ b/mypy/test/data.py
@@ -105,6 +105,12 @@ def parse_test_case(case: DataDrivenTestCase) -> None:
             src_path = join(os.path.dirname(case.file), item.arg)
             with open(src_path, encoding="utf8") as f:
                 files.append((join(base_path, "typing.pyi"), f.read()))
+        elif item.id == "_typeshed":
+            # Use an alternative stub file for the _typeshed module.
+            assert item.arg is not None
+            src_path = join(os.path.dirname(case.file), item.arg)
+            with open(src_path, encoding="utf8") as f:
+                files.append((join(base_path, "_typeshed.pyi"), f.read()))
         elif re.match(r"stale[0-9]*$", item.id):
             passnum = 1 if item.id == "stale" else int(item.id[len("stale") :])
             assert passnum > 0
diff --git a/mypy/test/testdeps.py b/mypy/test/testdeps.py
--- a/mypy/test/testdeps.py
+++ b/mypy/test/testdeps.py
@@ -50,7 +50,7 @@ def run_case(self, testcase: DataDrivenTestCase) -> None:
             type_state.add_all_protocol_deps(deps)
 
             for source, targets in sorted(deps.items()):
-                if source.startswith(("<enum", "<typing", "<mypy")):
+                if source.startswith(("<enum", "<typing", "<mypy", "<_typeshed.")):
                     # Remove noise.
                     continue
                 line = f"{source} -> {', '.join(sorted(targets))}"
diff --git a/test-data/unit/check-expressions.test b/test-data/unit/check-expressions.test
--- a/test-data/unit/check-expressions.test
+++ b/test-data/unit/check-expressions.test
@@ -1786,13 +1786,42 @@ b = {'z': 26, *a}  # E: invalid syntax
 
 [case testDictWithStarStarExpr]
 
-from typing import Dict
+from typing import Dict, Iterable
+
+class Thing:
+    def keys(self) -> Iterable[str]:
+        ...
+    def __getitem__(self, key: str) -> int:
+        ...
+
 a = {'a': 1}
 b = {'z': 26, **a}
 c = {**b}
 d = {**a, **b, 'c': 3}
-e = {1: 'a', **a}  # E: Argument 1 to "update" of "dict" has incompatible type "Dict[str, int]"; expected "Mapping[int, str]"
-f = {**b}  # type: Dict[int, int]  # E: List item 0 has incompatible type "Dict[str, int]"; expected "Mapping[int, int]"
+e = {1: 'a', **a}  # E: Argument 1 to "update" of "dict" has incompatible type "Dict[str, int]"; expected "SupportsKeysAndGetItem[int, str]"
+f = {**b}  # type: Dict[int, int]  # E: List item 0 has incompatible type "Dict[str, int]"; expected "SupportsKeysAndGetItem[int, int]"
+g = {**Thing()}
+h = {**a, **Thing()}
+i = {**Thing()}  # type: Dict[int, int]  # E: List item 0 has incompatible type "Thing"; expected "SupportsKeysAndGetItem[int, int]" \
+                 # N: Following member(s) of "Thing" have conflicts: \
+                 # N:     Expected: \
+                 # N:         def __getitem__(self, int, /) -> int \
+                 # N:     Got: \
+                 # N:         def __getitem__(self, str, /) -> int \
+                 # N:     Expected: \
+                 # N:         def keys(self) -> Iterable[int] \
+                 # N:     Got: \
+                 # N:         def keys(self) -> Iterable[str]
+j = {1: 'a', **Thing()}  # E: Argument 1 to "update" of "dict" has incompatible type "Thing"; expected "SupportsKeysAndGetItem[int, str]" \
+                         # N: Following member(s) of "Thing" have conflicts: \
+                         # N:     Expected: \
+                         # N:         def __getitem__(self, int, /) -> str \
+                         # N:     Got: \
+                         # N:         def __getitem__(self, str, /) -> int \
+                         # N:     Expected: \
+                         # N:         def keys(self) -> Iterable[int] \
+                         # N:     Got: \
+                         # N:         def keys(self) -> Iterable[str]
 [builtins fixtures/dict.pyi]
 [typing fixtures/typing-medium.pyi]
 
diff --git a/test-data/unit/check-formatting.test b/test-data/unit/check-formatting.test
--- a/test-data/unit/check-formatting.test
+++ b/test-data/unit/check-formatting.test
@@ -125,14 +125,29 @@ b'%(x)s' % {b'x': b'data'}
 [typing fixtures/typing-medium.pyi]
 
 [case testStringInterpolationMappingDictTypes]
-from typing import Any, Dict
+from typing import Any, Dict, Iterable
+
+class StringThing:
+    def keys(self) -> Iterable[str]:
+        ...
+    def __getitem__(self, __key: str) -> str:
+        ...
+
+class BytesThing:
+    def keys(self) -> Iterable[bytes]:
+        ...
+    def __getitem__(self, __key: bytes) -> str:
+        ...
+
 a = None # type: Any
 ds, do, di = None, None, None # type: Dict[str, int], Dict[object, int], Dict[int, int]
-'%(a)' % 1  # E: Format requires a mapping (expression has type "int", expected type for mapping is "Mapping[str, Any]")
+'%(a)' % 1  # E: Format requires a mapping (expression has type "int", expected type for mapping is "SupportsKeysAndGetItem[str, Any]")
 '%()d' % a
 '%()d' % ds
-'%()d' % do  # E: Format requires a mapping (expression has type "Dict[object, int]", expected type for mapping is "Mapping[str, Any]")
-b'%()d' % ds  # E: Format requires a mapping (expression has type "Dict[str, int]", expected type for mapping is "Mapping[bytes, Any]")
+'%()d' % do  # E: Format requires a mapping (expression has type "Dict[object, int]", expected type for mapping is "SupportsKeysAndGetItem[str, Any]")
+b'%()d' % ds  # E: Format requires a mapping (expression has type "Dict[str, int]", expected type for mapping is "SupportsKeysAndGetItem[bytes, Any]")
+'%()s' % StringThing()
+b'%()s' % BytesThing()
 [builtins fixtures/primitives.pyi]
 
 [case testStringInterpolationMappingInvalidSpecifiers]
diff --git a/test-data/unit/check-generic-subtyping.test b/test-data/unit/check-generic-subtyping.test
--- a/test-data/unit/check-generic-subtyping.test
+++ b/test-data/unit/check-generic-subtyping.test
@@ -990,6 +990,7 @@ main:13: note: Revealed type is "builtins.dict[builtins.int, builtins.str]"
 main:14: error: Keywords must be strings
 main:14: error: Argument 1 to "func_with_kwargs" has incompatible type "**X1[str, int]"; expected "int"
 [builtins fixtures/dict.pyi]
+[typing fixtures/typing-medium.pyi]
 
 [case testSubtypingMappingUnpacking3]
 from typing import Generic, TypeVar, Mapping, Iterable
diff --git a/test-data/unit/check-incremental.test b/test-data/unit/check-incremental.test
--- a/test-data/unit/check-incremental.test
+++ b/test-data/unit/check-incremental.test
@@ -3699,8 +3699,8 @@ cache_fine_grained = False
 [file mypy.ini.2]
 \[mypy]
 cache_fine_grained = True
-[rechecked a, builtins, typing]
-[stale a, builtins, typing]
+[rechecked _typeshed, a, builtins, typing]
+[stale _typeshed, a, builtins, typing]
 [builtins fixtures/tuple.pyi]
 
 [case testIncrementalPackageNameOverload]
@@ -3751,8 +3751,8 @@ Signature: 8a477f597d28d172789f06886806bc55
 [file b.py.2]
 # uh
 -- Every file should get reloaded, since the cache was invalidated
-[stale a, b, builtins, typing]
-[rechecked a, b, builtins, typing]
+[stale _typeshed, a, b, builtins, typing]
+[rechecked _typeshed, a, b, builtins, typing]
 [builtins fixtures/tuple.pyi]
 
 [case testIncrementalBustedFineGrainedCache2]
@@ -3764,8 +3764,8 @@ import b
 [file b.py.2]
 # uh
 -- Every file should get reloaded, since the settings changed
-[stale a, b, builtins, typing]
-[rechecked a, b, builtins, typing]
+[stale _typeshed, a, b, builtins, typing]
+[rechecked _typeshed, a, b, builtins, typing]
 [builtins fixtures/tuple.pyi]
 
 [case testIncrementalBustedFineGrainedCache3]
@@ -3780,8 +3780,8 @@ import b
 [file b.py.2]
 # uh
 -- Every file should get reloaded, since the cache was invalidated
-[stale a, b, builtins, typing]
-[rechecked a, b, builtins, typing]
+[stale _typeshed, a, b, builtins, typing]
+[rechecked _typeshed, a, b, builtins, typing]
 [builtins fixtures/tuple.pyi]
 
 [case testIncrementalWorkingFineGrainedCache]
diff --git a/test-data/unit/check-inference.test b/test-data/unit/check-inference.test
--- a/test-data/unit/check-inference.test
+++ b/test-data/unit/check-inference.test
@@ -1671,7 +1671,9 @@ a() # E: "Dict[str, int]" not callable
 
 [case testInferDictInitializedToEmptyUsingUpdateError]
 a = {}  # E: Need type annotation for "a" (hint: "a: Dict[<type>, <type>] = ...")
-a.update([1, 2])  # E: Argument 1 to "update" of "dict" has incompatible type "List[int]"; expected "Mapping[Any, Any]"
+a.update([1, 2])  # E: Argument 1 to "update" of "dict" has incompatible type "List[int]"; expected "SupportsKeysAndGetItem[Any, Any]" \
+                  # N: "list" is missing following "SupportsKeysAndGetItem" protocol member: \
+                  # N:     keys
 a()  # E: "Dict[Any, Any]" not callable
 [builtins fixtures/dict.pyi]
 
diff --git a/test-data/unit/check-kwargs.test b/test-data/unit/check-kwargs.test
--- a/test-data/unit/check-kwargs.test
+++ b/test-data/unit/check-kwargs.test
@@ -499,7 +499,7 @@ g(**{})
 
 [case testKeywordUnpackWithDifferentTypes]
 # https://github.com/python/mypy/issues/11144
-from typing import Dict, Generic, TypeVar, Mapping
+from typing import Dict, Generic, TypeVar, Mapping, Iterable
 
 T = TypeVar("T")
 T2 = TypeVar("T2")
@@ -516,6 +516,12 @@ class C(Generic[T, T2]):
 class D:
     ...
 
+class E:
+    def keys(self) -> Iterable[str]:
+        ...
+    def __getitem__(self, key: str) -> float:
+        ...
+
 def foo(**i: float) -> float:
     ...
 
@@ -523,7 +529,8 @@ a: A[str, str]
 b: B[str, str]
 c: C[str, float]
 d: D
-e = {"a": "b"}
+e: E
+f = {"a": "b"}
 
 foo(k=1.5)
 foo(**a)
@@ -531,6 +538,7 @@ foo(**b)
 foo(**c)
 foo(**d)
 foo(**e)
+foo(**f)
 
 # Correct:
 
@@ -544,9 +552,9 @@ foo(**good1)
 foo(**good2)
 foo(**good3)
 [out]
-main:29: error: Argument 1 to "foo" has incompatible type "**A[str, str]"; expected "float"
-main:30: error: Argument 1 to "foo" has incompatible type "**B[str, str]"; expected "float"
-main:31: error: Argument after ** must be a mapping, not "C[str, float]"
-main:32: error: Argument after ** must be a mapping, not "D"
-main:33: error: Argument 1 to "foo" has incompatible type "**Dict[str, str]"; expected "float"
+main:36: error: Argument 1 to "foo" has incompatible type "**A[str, str]"; expected "float"
+main:37: error: Argument 1 to "foo" has incompatible type "**B[str, str]"; expected "float"
+main:38: error: Argument after ** must be a mapping, not "C[str, float]"
+main:39: error: Argument after ** must be a mapping, not "D"
+main:41: error: Argument 1 to "foo" has incompatible type "**Dict[str, str]"; expected "float"
 [builtins fixtures/dict.pyi]
diff --git a/test-data/unit/fine-grained-dataclass-transform.test b/test-data/unit/fine-grained-dataclass-transform.test
--- a/test-data/unit/fine-grained-dataclass-transform.test
+++ b/test-data/unit/fine-grained-dataclass-transform.test
@@ -86,9 +86,9 @@ class A(Dataclass):
 
 [out]
 main:7: error: Unexpected keyword argument "x" for "B"
-builtins.pyi:12: note: "B" defined here
+builtins.pyi:13: note: "B" defined here
 main:7: error: Unexpected keyword argument "y" for "B"
-builtins.pyi:12: note: "B" defined here
+builtins.pyi:13: note: "B" defined here
 ==
 
 [case frozenInheritanceViaDefault]
diff --git a/test-data/unit/fine-grained-modules.test b/test-data/unit/fine-grained-modules.test
--- a/test-data/unit/fine-grained-modules.test
+++ b/test-data/unit/fine-grained-modules.test
@@ -1279,12 +1279,12 @@ a.py:2: error: Too many arguments for "foo"
 
 [case testAddModuleAfterCache3-only_when_cache]
 # cmd: mypy main a.py
-# cmd2: mypy main a.py b.py c.py d.py e.py f.py g.py h.py
-# cmd3: mypy main a.py b.py c.py d.py e.py f.py g.py h.py
+# cmd2: mypy main a.py b.py c.py d.py e.py f.py g.py h.py i.py j.py
+# cmd3: mypy main a.py b.py c.py d.py e.py f.py g.py h.py i.py j.py
 # flags: --ignore-missing-imports --follow-imports=skip
 import a
 [file a.py]
-import b, c, d, e, f, g, h
+import b, c, d, e, f, g, h, i, j
 b.foo(10)
 [file b.py.2]
 def foo() -> None: pass
@@ -1294,6 +1294,8 @@ def foo() -> None: pass
 [file f.py.2]
 [file g.py.2]
 [file h.py.2]
+[file i.py.2]
+[file j.py.2]
 
 -- No files should be stale or reprocessed in the first step since the large number
 -- of missing files will force build to give up on cache loading.
diff --git a/test-data/unit/fine-grained.test b/test-data/unit/fine-grained.test
--- a/test-data/unit/fine-grained.test
+++ b/test-data/unit/fine-grained.test
@@ -7546,7 +7546,7 @@ def d() -> Dict[int, int]: pass
 [builtins fixtures/dict.pyi]
 [out]
 ==
-main:5: error: Argument 1 to "update" of "dict" has incompatible type "Dict[int, int]"; expected "Mapping[int, str]"
+main:5: error: Argument 1 to "update" of "dict" has incompatible type "Dict[int, int]"; expected "SupportsKeysAndGetItem[int, str]"
 
 [case testAwaitAndAsyncDef-only_when_nocache]
 from a import g