Repository: python/mypy

Issue:
0.990 regression with implicit complex/int promotion
Given this code:
```python
from typing import reveal_type
x: complex = 1
reveal_type(x)
if isinstance(x, int):
    reveal_type(x)
```
mypy 0.990:
```
% mypy --strict --warn-unreachable ~/py/tmp/compl.py
/Users/jelle/py/tmp/compl.py:3: note: Revealed type is "builtins.complex"
/Users/jelle/py/tmp/compl.py:4: error: Subclass of "complex" and "int" cannot exist: would have incompatible method signatures  [unreachable]
/Users/jelle/py/tmp/compl.py:5: error: Statement is unreachable  [unreachable]
Found 2 errors in 1 file (checked 1 source file)
```

mypy 0.982:
```
% mypy --strict --warn-unreachable ~/py/tmp/compl.py
/Users/jelle/py/tmp/compl.py:3: note: Revealed type is "builtins.complex"
/Users/jelle/py/tmp/compl.py:5: note: Revealed type is "builtins.int"
Success: no issues found in 1 source file
```

The 0.982 behavior is correct, since `int` is a subclass of `complex` in the static type system.


Discussion:
This bug introduces 8 false positives when mypy checks the [`flake8-pyi` codebase](https://github.com/PyCQA/flake8-pyi/blob/master/pyi.py) (we have `--warn-unreachable` enabled in our `pyproject.toml` file):

<details>

```
pyi.py:1126: error: Subclass of "complex" and "int" cannot exist: would
have incompatible method signatures  [unreachable]
                    if type(literal.n) is int:
                            ^~~~~~~~~
pyi.py:1275: error: Subclass of "complex" and "int" cannot exist: would
have incompatible method signatures  [unreachable]
                    if isinstance(slc.n, int) and slc.n == 0:
                                  ^~~~~
pyi.py:1275: error: Right operand of "and" is never evaluated
[unreachable]
                    if isinstance(slc.n, int) and slc.n == 0:
                                                  ^~~~~~~~~~
pyi.py:1276: error: Statement is unreachable  [unreachable]
                        must_be_single = True
                        ^~~~~~~~~~~~~~~~~~~~~
pyi.py:1287: error: Subclass of "complex" and "int" cannot exist: would
have incompatible method signatures  [unreachable]
                        and isinstance(slc.upper.n, int)
                                       ^~~~~~~~~~~
pyi.py:1288: error: Right operand of "and" is never evaluated
[unreachable]
                        and slc.upper.n in (1, 2)
                            ^~~~~~~~~~~~~~~~~~~~~
pyi.py:1290: error: Statement is unreachable  [unreachable]
                        can_have_strict_equals = slc.upper.n
                        ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
pyi.py:1313: error: Subclass of "complex" and "int" cannot exist: would
have incompatible method signatures  [unreachable]
    ...nstance(comparator, ast.Num) or not isinstance(comparator.n, int):
                                                      ^~~~~~~~~~~~
Found 8 errors in 1 file (checked 1 source file)
```

</details>
cc @ilevkivskyi this is from https://github.com/python/mypy/issues/13781
FWIW this is intentional, `isinstance()` checks have always been determined by runtime behavior, not static subtyping. Previously promotions sometimes leaked, and caused various inconsistencies.

There is really no middle ground, if we enable promotions, there will be other issues, see, e.g. https://github.com/python/mypy/issues/6060. I would say current behavior is better because:
* It is more consistent with how `isinstance()` check behaves in other situations
* It may cause "false positives" with a strictness flag, while opposite causes false positives with default options.
Also fix should be trivial, use a more precise type, like `Union[int, complex]`.
Without strictness flags, the current behavior leads code to be silently skipped as unreachable, which is arguably worse. I agree that there are few good options here though.
Maybe there's some hack, so we keep the better narrowing behaviour, but it doesn't affect reachability
@hauntsaninja Actually, if you like hacks, I think there may be a (relatively) easy one: ignore the impossible intersection check in case if one type can be promoted to another one.
Well, we should probably still prohibit (consider unreachable) something like:
```python
if isinstance(x, int) and isinstance(x, float):
    ...
```
so it may require some work to implement.
Just making sure you all are aware of the idea in #11516

Specifically: in the if statement, split types up into this union and rely on already-existing narrowing.

Y'all are probably already aware, but just making sure this "solution" is considered. (I don't see any reasoning on that issue itself that would disqualify it)
@hauntsaninja Actually after thinking some more, it seems to me we shouldn't change this, otherwise we may not flag legitimately unreachable code like
```python
x = 1j
if isinstance(x, int):
    ...
```
that may potentially also hide bugs.

In general, most of the original motivation for type promotions was that practically all functions originally intended for `float` will also (accidentally) work for `int`. So to avoid forcing people to annotate all their argument types as e.g. `Union[int, float]`, this decision was made. On the opposite, if a function actually differentiates between `int` and `float`, it should be annotated as taking `Union[int, float]` to emphasize this:
```python
def weighted(a: float, b: float) -> float:
    return (a + 2 * b) / 3
weighted(1, 2)  # This is fine

def num_type(x: float) -> str:  # Bad, should be Union[int, float]
    if isinstance(x, int):
        return "Integer"
    return "Floating"
```
Anyway, it is still gray area and likely there will still be some people who likes (or used to) the old behavior. So we should clearly document this.
I agree with Ivan's proposed approach. Some thoughts on what we'd need to change:

- In mypy, the new behavior can lead mypy to silently skip checking code. To make that a less likely problem, we should move forward with including `--warn-unreachable` in `--strict` (#11223).
- In flake8-pyi we should drop the check against `int | float` unions (Y041).
- In typeshed we should document when to use `int | float` and make changes where appropriate, including in the type for `ast.Num.n`.
Some thoughts from me:

- I find it pretty confusing for us to say "`int` is an implicit subtype of `float`, but only sometimes. Sometimes you can just annotate an argument with `float` and it'll mean `float | int`, but other times it'll cause surprising errors." Rereading PEP 484, the spec is vaguer than I remembered, so I guess this isn't going against the spec. It feels inherently confusing and pretty user-unfriendly, though, and I'm not sure any amount of documentation is going to solve that :(
- I'm not exactly a fan of implicit promotions; in an ideal world, I'd love it if we could have a more principled solution to this problem. But deprecating `float/int` subtyping doesn't feel like a realistic proposal at this point in time.
- I agree that we should definitely change flake8-pyi's Y041 check if mypy is sticking with this behaviour. I'm not sure if we should drop it completely or change it so that it only applies to parameter annotations in functions.
- Making `--warn-unreachable` the default if you have `--strict` enabled would be great, but I don't think it's doable at the moment. It currently has loads of false positives if you're dealing with code that has platform-specific branches. You can see this if you try to enable `--warn-unreachable` for mypy's selfcheck.
I think that it would be least confusing if we managed to preserve the `isinstance` behavior of 0.982 while still ignoring promotions when simplifying unions generally. 

I think that it's okay to allow users to be explicit and annotate things as `float | int` and so on, but this feels like a pretty advanced feature and most users should need to know about it.

> ... that may potentially also hide bugs.

I don't think that is a significant issue. This is also possible with real subclasses and I don't think anybody has complained about code like this not generating errors:

```py
class Base: pass
class Sub(Base): pass

o = Base()
if isinstance(o, Sub): 
    <something>
```
> I think that it would be least confusing if we managed to preserve the isinstance behavior of 0.982 while still ignoring promotions when simplifying unions generally.

`isinstance()` behavior in 0.982 is just wrong, and we had issues about it, examples https://github.com/python/mypy/issues/6060, https://github.com/python/mypy/issues/12824
> This is also possible with real subclasses and I don't think anybody has complained about code like this not generating errors

This is different. With `complex` vs `int` that branch is always unreachable, and if someone asks for `--warn-unreachable` we should warn. Not doing so is a bug.
> I find it pretty confusing for us to say "int is an implicit subtype of float, but only sometimes...

It was always like this, since both extremes are clearly bad, we need to find a reasonable/clear midpoint. FWIW the current one seems more consistent to me than in 0.982
Also in general I am in favor of adding `--warn-unreachable` to `--strict`, but as it was mentioned above, this will require some work.
> isinstance() behavior in 0.982 is just wrong, and we had issues about it, examples https://github.com/python/mypy/issues/6060, https://github.com/python/mypy/issues/12824

I don't want to resurrect those bugs. But I'd specifically like this one to not generate unreachable code:
```py
from typing import reveal_type
x: complex = 1
reveal_type(x)
if isinstance(x, int):
    reveal_type(x)
```
Requiring `x: complex | int = 1` here (but not generally) seems hard to explain, and having the body of the if statement unreachable could result in buggy code getting accepted if `--warn-unreachable` is not used. I may be okay with some special casing for this use case. Uses may also consider the 0.990 behavior a regression from 0.982.

> This is different. With complex vs int that branch is always unreachable, and if someone asks for --warn-unreachable we should warn. Not doing so is a bug.

I think that sometimes not detecting unreachable code is less of a problem than code that is silently not being type checked because mypy things that it's unreachable but it really is reachable. I'd expect that the vast majority of users aren't using `--warn-unreachable`.
> I don't want to resurrect those bugs. But I'd specifically like this one to not generate unreachable code

OK, as I said above it is possible with some special casing. The less easy part is how to make the special-casing consistent. e.g. should these two be flagged as unreachable?
```python
x: object  # Or a union
if isinstance(x, complex) and isinstance(x, int):
    ...

if isinsatnce(x, complex):
    if isinstance(x, int):
        ...
```
If we continue to flag them as unreachable, then it will be inconsistent with
```python
x: complex
if isinsatnce(x, int):
    ...
```
If we will also make those two reachable, what type will we give to `x` inside? An impossible intersection, or make it depend on order `int` vs `complex`?
Actually, maybe saying _all_ of those are reachable, and `x` having `int` type (even with flipped order) is an option.
> Actually, maybe saying all of those are reachable, and x having int type (even with flipped order) is an option.

It seems reasonable to have all of the examples reachable. This could be either way (we can go with whatever is straightforward to implement):
```py
x: object  # Or a union
if isinsatnce(x, int):
    if isinstance(x, complex):
        ...  # Unreachable?
```

Resolving patch:
diff --git a/mypy/checker.py b/mypy/checker.py
--- a/mypy/checker.py
+++ b/mypy/checker.py
@@ -4823,7 +4823,7 @@ def make_fake_typeinfo(
         return cdef, info
 
     def intersect_instances(
-        self, instances: tuple[Instance, Instance], ctx: Context
+        self, instances: tuple[Instance, Instance], errors: list[tuple[str, str]]
     ) -> Instance | None:
         """Try creating an ad-hoc intersection of the given instances.
 
@@ -4850,6 +4850,17 @@ def intersect_instances(
         curr_module = self.scope.stack[0]
         assert isinstance(curr_module, MypyFile)
 
+        # First, retry narrowing while allowing promotions (they are disabled by default
+        # for isinstance() checks, etc). This way we will still type-check branches like
+        # x: complex = 1
+        # if isinstance(x, int):
+        #     ...
+        left, right = instances
+        if is_proper_subtype(left, right, ignore_promotions=False):
+            return left
+        if is_proper_subtype(right, left, ignore_promotions=False):
+            return right
+
         def _get_base_classes(instances_: tuple[Instance, Instance]) -> list[Instance]:
             base_classes_ = []
             for inst in instances_:
@@ -4890,17 +4901,10 @@ def _make_fake_typeinfo_and_full_name(
                     self.check_multiple_inheritance(info)
             info.is_intersection = True
         except MroError:
-            if self.should_report_unreachable_issues():
-                self.msg.impossible_intersection(
-                    pretty_names_list, "inconsistent method resolution order", ctx
-                )
+            errors.append((pretty_names_list, "inconsistent method resolution order"))
             return None
-
         if local_errors.has_new_errors():
-            if self.should_report_unreachable_issues():
-                self.msg.impossible_intersection(
-                    pretty_names_list, "incompatible method signatures", ctx
-                )
+            errors.append((pretty_names_list, "incompatible method signatures"))
             return None
 
         curr_module.names[full_name] = SymbolTableNode(GDEF, info)
@@ -6345,15 +6349,20 @@ def conditional_types_with_intersection(
             possible_target_types.append(item)
 
         out = []
+        errors: list[tuple[str, str]] = []
         for v in possible_expr_types:
             if not isinstance(v, Instance):
                 return yes_type, no_type
             for t in possible_target_types:
-                intersection = self.intersect_instances((v, t), ctx)
+                intersection = self.intersect_instances((v, t), errors)
                 if intersection is None:
                     continue
                 out.append(intersection)
         if len(out) == 0:
+            # Only report errors if no element in the union worked.
+            if self.should_report_unreachable_issues():
+                for types, reason in errors:
+                    self.msg.impossible_intersection(types, reason, ctx)
             return UninhabitedType(), expr_type
         new_yes_type = make_simplified_union(out)
         return new_yes_type, expr_type
diff --git a/mypy/join.py b/mypy/join.py
--- a/mypy/join.py
+++ b/mypy/join.py
@@ -141,8 +141,11 @@ def join_instances_via_supertype(self, t: Instance, s: Instance) -> ProperType:
 
 
 def join_simple(declaration: Type | None, s: Type, t: Type) -> ProperType:
-    """Return a simple least upper bound given the declared type."""
-    # TODO: check infinite recursion for aliases here?
+    """Return a simple least upper bound given the declared type.
+
+    This function should be only used by binder, and should not recurse.
+    For all other uses, use `join_types()`.
+    """
     declaration = get_proper_type(declaration)
     s = get_proper_type(s)
     t = get_proper_type(t)
@@ -158,10 +161,10 @@ def join_simple(declaration: Type | None, s: Type, t: Type) -> ProperType:
     if isinstance(s, ErasedType):
         return t
 
-    if is_proper_subtype(s, t):
+    if is_proper_subtype(s, t, ignore_promotions=True):
         return t
 
-    if is_proper_subtype(t, s):
+    if is_proper_subtype(t, s, ignore_promotions=True):
         return s
 
     if isinstance(declaration, UnionType):
@@ -176,6 +179,9 @@ def join_simple(declaration: Type | None, s: Type, t: Type) -> ProperType:
     # Meets/joins require callable type normalization.
     s, t = normalize_callables(s, t)
 
+    if isinstance(s, UnionType) and not isinstance(t, UnionType):
+        s, t = t, s
+
     value = t.accept(TypeJoinVisitor(s))
     if declaration is None or is_subtype(value, declaration):
         return value
diff --git a/mypy/meet.py b/mypy/meet.py
--- a/mypy/meet.py
+++ b/mypy/meet.py
@@ -124,7 +124,15 @@ def narrow_declared_type(declared: Type, narrowed: Type) -> Type:
             [
                 narrow_declared_type(x, narrowed)
                 for x in declared.relevant_items()
-                if is_overlapping_types(x, narrowed, ignore_promotions=True)
+                # This (ugly) special-casing is needed to support checking
+                # branches like this:
+                # x: Union[float, complex]
+                # if isinstance(x, int):
+                #     ...
+                if (
+                    is_overlapping_types(x, narrowed, ignore_promotions=True)
+                    or is_subtype(narrowed, x, ignore_promotions=False)
+                )
             ]
         )
     if is_enum_overlapping_union(declared, narrowed):


Test patch:
diff --git a/test-data/unit/check-classes.test b/test-data/unit/check-classes.test
--- a/test-data/unit/check-classes.test
+++ b/test-data/unit/check-classes.test
@@ -7209,7 +7209,7 @@ from typing import Callable
 class C:
     x: Callable[[C], int] = lambda x: x.y.g()  # E: "C" has no attribute "y"
 
-[case testOpWithInheritedFromAny]
+[case testOpWithInheritedFromAny-xfail]
 from typing import Any
 C: Any
 class D(C):
diff --git a/test-data/unit/check-isinstance.test b/test-data/unit/check-isinstance.test
--- a/test-data/unit/check-isinstance.test
+++ b/test-data/unit/check-isinstance.test
@@ -2392,7 +2392,7 @@ class B:
             x1: Literal[1] = self.f()
 
     def t2(self) -> None:
-        if isinstance(self, (A0, A1)):  # E: Subclass of "B" and "A0" cannot exist: would have incompatible method signatures
+        if isinstance(self, (A0, A1)):
             reveal_type(self)           # N: Revealed type is "__main__.<subclass of "A1" and "B">1"
             x0: Literal[0] = self.f()   # E: Incompatible types in assignment (expression has type "Literal[1]", variable has type "Literal[0]")
             x1: Literal[1] = self.f()
diff --git a/test-data/unit/check-type-promotion.test b/test-data/unit/check-type-promotion.test
--- a/test-data/unit/check-type-promotion.test
+++ b/test-data/unit/check-type-promotion.test
@@ -54,3 +54,136 @@ def f(x: Union[SupportsFloat, T]) -> Union[SupportsFloat, T]: pass
 f(0)  # should not crash
 [builtins fixtures/primitives.pyi]
 [out]
+
+[case testIntersectionUsingPromotion1]
+# flags: --warn-unreachable
+from typing import Union
+
+x: complex = 1
+reveal_type(x)  # N: Revealed type is "builtins.complex"
+if isinstance(x, int):
+    reveal_type(x)  # N: Revealed type is "builtins.int"
+else:
+    reveal_type(x)  # N: Revealed type is "builtins.complex"
+reveal_type(x)  # N: Revealed type is "builtins.complex"
+
+y: Union[int, float]
+if isinstance(y, float):
+    reveal_type(y)  # N: Revealed type is "builtins.float"
+else:
+    reveal_type(y)  # N: Revealed type is "builtins.int"
+
+reveal_type(y)  # N: Revealed type is "Union[builtins.int, builtins.float]"
+
+if isinstance(y, int):
+    reveal_type(y)  # N: Revealed type is "builtins.int"
+else:
+    reveal_type(y)  # N: Revealed type is "builtins.float"
+[builtins fixtures/primitives.pyi]
+
+[case testIntersectionUsingPromotion2]
+# flags: --warn-unreachable
+x: complex = 1
+reveal_type(x)  # N: Revealed type is "builtins.complex"
+if isinstance(x, (int, float)):
+    reveal_type(x)  # N: Revealed type is "Union[builtins.int, builtins.float]"
+else:
+    reveal_type(x)  # N: Revealed type is "builtins.complex"
+
+# Note we make type precise, since type promotions are involved
+reveal_type(x)  # N: Revealed type is "Union[builtins.complex, builtins.int, builtins.float]"
+[builtins fixtures/primitives.pyi]
+
+[case testIntersectionUsingPromotion3]
+# flags: --warn-unreachable
+x: object
+if isinstance(x, int) and isinstance(x, complex):
+    reveal_type(x)  # N: Revealed type is "builtins.int"
+if isinstance(x, complex) and isinstance(x, int):
+    reveal_type(x)  # N: Revealed type is "builtins.int"
+[builtins fixtures/primitives.pyi]
+
+[case testIntersectionUsingPromotion4]
+# flags: --warn-unreachable
+x: object
+if isinstance(x, int):
+    if isinstance(x, complex):
+        reveal_type(x)  # N: Revealed type is "builtins.int"
+    else:
+        reveal_type(x)  # N: Revealed type is "builtins.int"
+if isinstance(x, complex):
+    if isinstance(x, int):
+        reveal_type(x)  # N: Revealed type is "builtins.int"
+    else:
+        reveal_type(x)  # N: Revealed type is "builtins.complex"
+[builtins fixtures/primitives.pyi]
+
+[case testIntersectionUsingPromotion5]
+# flags: --warn-unreachable
+from typing import Union
+
+x: Union[float, complex]
+if isinstance(x, int):
+    reveal_type(x)  # N: Revealed type is "builtins.int"
+else:
+    reveal_type(x)  # N: Revealed type is "Union[builtins.float, builtins.complex]"
+reveal_type(x)  # N: Revealed type is "Union[builtins.int, builtins.float, builtins.complex]"
+[builtins fixtures/primitives.pyi]
+
+[case testIntersectionUsingPromotion6]
+# flags: --warn-unreachable
+from typing import Union
+
+x: Union[str, complex]
+if isinstance(x, int):
+    reveal_type(x)  # N: Revealed type is "builtins.int"
+else:
+    reveal_type(x)  # N: Revealed type is "Union[builtins.str, builtins.complex]"
+reveal_type(x)  # N: Revealed type is "Union[builtins.str, builtins.int, builtins.complex]"
+[builtins fixtures/primitives.pyi]
+
+[case testIntersectionUsingPromotion7]
+# flags: --warn-unreachable
+from typing import Union
+
+x: Union[int, float, complex]
+if isinstance(x, int):
+    reveal_type(x)  # N: Revealed type is "builtins.int"
+else:
+    reveal_type(x)  # N: Revealed type is "Union[builtins.float, builtins.complex]"
+
+reveal_type(x)  # N: Revealed type is "Union[builtins.int, builtins.float, builtins.complex]"
+
+if isinstance(x, float):
+    reveal_type(x)  # N: Revealed type is "builtins.float"
+else:
+    reveal_type(x)  # N: Revealed type is "Union[builtins.int, builtins.complex]"
+
+reveal_type(x)  # N: Revealed type is "Union[builtins.int, builtins.float, builtins.complex]"
+
+if isinstance(x, complex):
+    reveal_type(x)  # N: Revealed type is "builtins.complex"
+else:
+    reveal_type(x)  # N: Revealed type is "Union[builtins.int, builtins.float]"
+
+reveal_type(x)  # N: Revealed type is "Union[builtins.int, builtins.float, builtins.complex]"
+[builtins fixtures/primitives.pyi]
+
+[case testIntersectionUsingPromotion8]
+# flags: --warn-unreachable
+from typing import Union
+
+x: Union[int, float, complex]
+if isinstance(x, (int, float)):
+    reveal_type(x)  # N: Revealed type is "Union[builtins.int, builtins.float]"
+else:
+    reveal_type(x)  # N: Revealed type is "builtins.complex"
+if isinstance(x, (int, complex)):
+    reveal_type(x)  # N: Revealed type is "Union[builtins.int, builtins.complex]"
+else:
+    reveal_type(x)  # N: Revealed type is "builtins.float"
+if isinstance(x, (float, complex)):
+    reveal_type(x)  # N: Revealed type is "Union[builtins.float, builtins.complex]"
+else:
+    reveal_type(x)  # N: Revealed type is "builtins.int"
+[builtins fixtures/primitives.pyi]