Repository: python/mypy

Issue:
`TypeGuard` doesn't work with `TypeVar`
**Bug Report**

Using a `TypeVar` with `TypeGuard` results in Mypy thinking that the resulting type is the `TypeVar` itself, and not the type it should be pointing to.

**To Reproduce**

Example adapted from [PEP 647](https://www.python.org/dev/peps/pep-0647/#typeguard-type):
```py
# src/test.py

from typing import TypeGuard, TypeVar

_T = TypeVar("_T")


def is_two_element_tuple(val: tuple[_T, ...]) -> TypeGuard[tuple[_T, _T]]:
    return len(val) == 2


def func(names: tuple[str, ...]):
    if is_two_element_tuple(names):
        reveal_type(names)  # Tuple[str, str]
        for name in names:
            print(name.capitalize(), "- two")
    else:
        reveal_type(names)  # Tuple[str, ...]
        for name in names:
            print(name.capitalize(), "- not two")

```

**Expected Behavior**

Mypy should substitute the `TypeVar` inside `TypeGuard` to the proper type it should be pointing to.
Pyre, Pyright, and Pylance do successfully work with the `TypeVar` inside `TypeGuard`.

```
$ pyre
ƛ No type errors found
src/test.py:14:8 Revealed type [-1]: Revealed type for `names` is `typing.Tuple[str, str]`.
src/test.py:18:8 Revealed type [-1]: Revealed type for `names` is `typing.Tuple[str, ...]`.

$ pyright src/test.py
/home/micael/projects/python-3_10/src/test.py
  /home/micael/projects/python-3_10/src/test.py:14:21 - info: Type of "names" is "tuple[str, str]"
  /home/micael/projects/python-3_10/src/test.py:18:21 - info: Type of "names" is "tuple[str, ...]"
0 errors, 0 warnings, 2 infos
```

**Actual Behavior**

```
$ mypy src/test.py
src/test.py:14: note: Revealed type is "Tuple[_T`-1, _T`-1]"
src/test.py:16: error: "_T" has no attribute "capitalize"
src/test.py:18: note: Revealed type is "builtins.tuple[builtins.str]"
src/test.py:19: error: Incompatible types in assignment (expression has type "str", variable has type "_T")
src/test.py:20: error: "_T" has no attribute "capitalize"
Found 3 errors in 1 file (checked 1 source file)
```

**Your Environment**

| Name | Version |
| -- | -- |
| Python | `3.10.0` |
| typing_extensions | `4.0.1` |
| mypy | `0.920` |
| mypy-extensions | `0.4.3` |
| pyre-check | `0.9.8` |
| pyre-extensions | `0.0.23` |
| pyright | `0.0.13` |

- Mypy version used: `0.920`
- Mypy command-line flags: none
- Mypy configuration options from `mypy.ini` (and other config files): none
- Python version used: `3.10.0`
- Operating system and version: Manjaro KDE

> Operating System: Manjaro Linux
KDE Plasma Version: 5.23.4
KDE Frameworks Version: 5.88.0
Qt Version: 5.15.2
Kernel Version: 5.15.7-1-MANJARO (64-bit)
Graphics Platform: X11
Processors: 4 × AMD Ryzen 3 2200G with Radeon Vega Graphics
Memory: 7.7 GiB of RAM
Graphics Processor: NVIDIA GeForce GTX 970/PCIe/SSE2
TypeGuard functions not properly recognizing generic types
**Bug Report**

It seems the `typing.TypeGuard` support is not able to properly discriminate a union of two (generic) types.

**To Reproduce**

* The example below defines two completely distinct types, `Left[L]` and `Right[R]`, both of which happen to be generic and have a `.value` attribute.
* The `LeftOrRight` is a `Union` of those types.
* `x` is a variable of type `LeftOrRight[str, int]`. In this example it has value `Left("")`
* Using `isinstance()` works fine to figure out whether `x` is either `Left` or `Right`, and within the `if/else` block the types are correct, including the types `L` and `R` (both `TypeVar`)
* The `is_left()` and `is_right()` helpers use `typing.TypeGuard`, but Mypy does not recognize the types like it did with the `isinstance()` check.

```python
from typing import Generic, TypeGuard, TypeVar, Union

L = TypeVar("L")
R = TypeVar("R")


class Left(Generic[L]):
    value: L

    def __init__(self, value: L):
        self.value = value


class Right(Generic[R]):
    value: R

    def __init__(self, value: R):
        self.value = value


LeftOrRight = Union[Left[L], Right[R]]


def is_left(obj: LeftOrRight[L, R]) -> TypeGuard[Left[L]]:
    return isinstance(obj, Left)


def is_right(obj: LeftOrRight[L, R]) -> TypeGuard[Right[R]]:
    return isinstance(obj, Right)


x: LeftOrRight[str, int] = Left("")
reveal_type(x)  # Revealed type is "Union[Left[builtins.str], Right[builtins.int]]"

if isinstance(x, Left):
    reveal_type(x)  # Revealed type is "Left[builtins.str]"
    x.value + ""  # only valid for str
else:
    reveal_type(x)  # Revealed type is "Right[builtins.int]"
    x.value + 10  # only valid for int

if is_left(x):
    reveal_type(x)  # Revealed type is "Left[L`-1]"
    x.value + ""  #  Unsupported left operand type for + ("L")  [operator]
else:
    reveal_type(x)  # Revealed type is "Union[Left[builtins.str], Right[builtins.int]]"
    x.value + 10  # Unsupported operand types for + ("str" and "int")  [operator]

```

**Expected Behavior**

`is_left()` and `is_right()` act as type-safe ways to discriminate between the two members of the `Union`, including the actual type of the (generic) `.value` attribute.

**Actual Behavior**

* `reveal_type()` returns wrong results
* type errors reported in what seems to be perfectly fine and very strict code

**Your Environment**

This happens with both Mypy 0.910 (current release as time of writing) and today's `git` checkout of the main branch.



Discussion:
i think this can be closed as a duplicate of #11428, @MicaelJarniac? (as you [noticed yourself](https://github.com/python/mypy/issues/11428#issuecomment-996894937))
I believe the bug in question is to do with `TypeVar`, and not `Generic`, as your issue implies.

And in my opinion, the examples I've given here are easier to understand and more straight to the point.

I'm not sure about what to do.

One thing I did not test is whether the issue I'm describing here also happens on Python 3.10, and that's why this issue is about the `TypeGuard` from `typing_extensions`.
But the issue you're having seems to be happening on at least Python 3.10.

I do think our issues are about the same bug, but we've come across it in different ways.

I'll try to test my examples against Python 3.10, to see whether it's in any way related to `typing_extensions` or not.
If it is still happening on 3.10, then I do think we could look into merging both issues onto one.
Just tested it with Python 3.10.0 and using the built-in `TypeGuard` from `typing`, and the bug is still there.
friendly ping 👋🏼 @hauntsaninja @ilevkivskyi, since you were involved with `TypeGuard` related issues fairly recently, e.g. #11061, #10899, #10647, #11007
The current behavior is correct and intended. User-defined type guards intentionally do not provide type narrowing for the negative ("else") case. To quote from PEP 647:

> User-defined type guards apply narrowing only in the positive case (the if clause). The type is not narrowed in the negative case.

This decision was based on experience with user-defined type guards in TypeScript. It supports the negative case, and this leads to problems in a number of real-world cases. The TypeScript authors admitted that if they able to go back in time, they would probably reverse their decision. We learned from this when drafting PEP 647.

Some built-in type guards (such as `isinstance` checks) can be safely narrowed in the negative case because the semantics are well known. In the case of user-defined type guards, the semantics depend on the use case, and a type checker cannot assume that positive and negative checks are symmetrical. There are many examples where they are not.

I was too hasty in my response. I think I misunderstood the problem you were reporting.

You are correct that mypy isn't correctly applying the solved type variables in the case of the type guard.

```python
    if is_left(x):
        reveal_type(x)  # Mypy: Left[L`-1], Pyright: List[str]
        x.value + ""
    elif is_right(x):
        reveal_type(x)  # Mypy: Left[str] | Right[int]], Pyright: Right[int]
        x.value + 10
```

thanks for the response, @erictraut 🙏🏼 

the lack of support for the negative case would account for the errors seen in the `else` clause of the example.

however, the positive case _also_ fails currently:

``` python
if is_left(x):
    reveal_type(x)  # Revealed type is "Left[L`-1]"
    x.value + ""  #  Unsupported left operand type for + ("L")  [operator]
```

... even though the type should be fully know at this point: `x` is either `Left[L]` or `Right[R]`, and the typeguard check should turn it into `Left[L]`. since the type of `L` is known, namely a `str`, i would expect that `x.value + ""` works without type errors.

the `reveal_type()` line seems to indicate this information is lost:

```
    reveal_type(x)  # Revealed type is "Left[L`-1]"
```

...which would make `TypeGuard` quite useless for generic types, specifically for things like [`Result`](https://github.com/rustedpy/result) (which is what triggered me to look into this in the first place).
Yes, I agree. You can address the negative case by adding an `elif` check like I did in my last response, but that still leaves a problem.
for full transparency: the problem i'm trying to solve is this one: https://github.com/rustedpy/result/issues/69

...and it seems that this is not possible to achieve currently.
I believe I've stumbled upon the same issue, but when using `TypeGuard` from `typing_extensions`: https://github.com/python/mypy/issues/11780

Resolving patch:
diff --git a/mypy/applytype.py b/mypy/applytype.py
--- a/mypy/applytype.py
+++ b/mypy/applytype.py
@@ -103,6 +103,12 @@ def apply_generic_arguments(
     # Apply arguments to argument types.
     arg_types = [expand_type(at, id_to_type) for at in callable.arg_types]
 
+    # Apply arguments to TypeGuard if any.
+    if callable.type_guard is not None:
+        type_guard = expand_type(callable.type_guard, id_to_type)
+    else:
+        type_guard = None
+
     # The callable may retain some type vars if only some were applied.
     remaining_tvars = [tv for tv in tvars if tv.id not in id_to_type]
 
@@ -110,4 +116,5 @@ def apply_generic_arguments(
         arg_types=arg_types,
         ret_type=expand_type(callable.ret_type, id_to_type),
         variables=remaining_tvars,
+        type_guard=type_guard,
     )
diff --git a/mypy/checkexpr.py b/mypy/checkexpr.py
--- a/mypy/checkexpr.py
+++ b/mypy/checkexpr.py
@@ -344,11 +344,6 @@ def visit_call_expr_inner(self, e: CallExpr, allow_none_return: bool = False) ->
                                         ret_type=self.object_type(),
                                         fallback=self.named_type('builtins.function'))
         callee_type = get_proper_type(self.accept(e.callee, type_context, always_allow_any=True))
-        if (isinstance(e.callee, RefExpr)
-                and isinstance(callee_type, CallableType)
-                and callee_type.type_guard is not None):
-            # Cache it for find_isinstance_check()
-            e.callee.type_guard = callee_type.type_guard
         if (self.chk.options.disallow_untyped_calls and
                 self.chk.in_checked_function() and
                 isinstance(callee_type, CallableType)
@@ -886,10 +881,19 @@ def check_call_expr_with_callee_type(self,
         # Unions are special-cased to allow plugins to act on each item in the union.
         elif member is not None and isinstance(object_type, UnionType):
             return self.check_union_call_expr(e, object_type, member)
-        return self.check_call(callee_type, e.args, e.arg_kinds, e,
-                               e.arg_names, callable_node=e.callee,
-                               callable_name=callable_name,
-                               object_type=object_type)[0]
+        ret_type, callee_type = self.check_call(
+            callee_type, e.args, e.arg_kinds, e,
+            e.arg_names, callable_node=e.callee,
+            callable_name=callable_name,
+            object_type=object_type,
+        )
+        proper_callee = get_proper_type(callee_type)
+        if (isinstance(e.callee, RefExpr)
+                and isinstance(proper_callee, CallableType)
+                and proper_callee.type_guard is not None):
+            # Cache it for find_isinstance_check()
+            e.callee.type_guard = proper_callee.type_guard
+        return ret_type
 
     def check_union_call_expr(self, e: CallExpr, object_type: UnionType, member: str) -> Type:
         """"Type check calling a member expression where the base type is a union."""


Test patch:
diff --git a/test-data/unit/check-typeguard.test b/test-data/unit/check-typeguard.test
--- a/test-data/unit/check-typeguard.test
+++ b/test-data/unit/check-typeguard.test
@@ -547,3 +547,53 @@ accepts_typeguard(with_typeguard_a)  # E: Argument 1 to "accepts_typeguard" has
 accepts_typeguard(with_typeguard_b)
 accepts_typeguard(with_typeguard_c)
 [builtins fixtures/tuple.pyi]
+
+[case testTypeGuardWithIdentityGeneric]
+from typing import TypeVar
+from typing_extensions import TypeGuard
+
+_T = TypeVar("_T")
+
+def identity(val: _T) -> TypeGuard[_T]:
+    pass
+
+def func1(name: _T):
+    reveal_type(name)  # N: Revealed type is "_T`-1"
+    if identity(name):
+        reveal_type(name)  # N: Revealed type is "_T`-1"
+
+def func2(name: str):
+    reveal_type(name)  # N: Revealed type is "builtins.str"
+    if identity(name):
+        reveal_type(name)  # N: Revealed type is "builtins.str"
+[builtins fixtures/tuple.pyi]
+
+[case testTypeGuardWithGenericInstance]
+from typing import TypeVar, List
+from typing_extensions import TypeGuard
+
+_T = TypeVar("_T")
+
+def is_list_of_str(val: _T) -> TypeGuard[List[_T]]:
+    pass
+
+def func(name: str):
+    reveal_type(name)  # N: Revealed type is "builtins.str"
+    if is_list_of_str(name):
+        reveal_type(name)  # N: Revealed type is "builtins.list[builtins.str]"
+[builtins fixtures/tuple.pyi]
+
+[case testTypeGuardWithTupleGeneric]
+from typing import TypeVar, Tuple
+from typing_extensions import TypeGuard
+
+_T = TypeVar("_T")
+
+def is_two_element_tuple(val: Tuple[_T, ...]) -> TypeGuard[Tuple[_T, _T]]:
+    pass
+
+def func(names: Tuple[str, ...]):
+    reveal_type(names)  # N: Revealed type is "builtins.tuple[builtins.str, ...]"
+    if is_two_element_tuple(names):
+        reveal_type(names)  # N: Revealed type is "Tuple[builtins.str, builtins.str]"
+[builtins fixtures/tuple.pyi]