Repository: python/mypy
Pull request: Fix 0.750 regression: union (non-)simplification should not affect inference
Fixes https://github.com/python/mypy/issues/8051

The fix is as discussed in the issue. I didn't find a test case for the union `actual`, but I think it makes sense to simplify it as well. Note: I add the `pythoneval` test just as a regression test, if you think it is not worth it, I can remove it.

Problem statement:
mypy 0.750 regression with asyncio.wait
Please consider the following:

```python
from asyncio import Future, wait
from typing import List

async def foo() -> None:
    f: List[Future[None]] = []
    await wait(f)
```

Checking this with mypy 0.740 (Python 3.8.0) without options succeeds. Checking it with mypy 0.750 (also Python 3.8.0, no options) prints the following error:

```
foo.py:6: error: Argument 1 to "wait" has incompatible type "List[Future[None]]"; expected "Iterable[Union[Future[<nothing>], Generator[Any, None, <nothing>], Awaitable[<nothing>]]]"
Found 1 error in 1 file (checked 1 source file)
```

The annotation of `asyncio.wait()` from typeshed has not changed between mypy versions:

```python
_T = TypeVar('_T')
...
_FutureT = Union[Future[_T], Generator[Any, None, _T], Awaitable[_T]]
...
def wait(fs: Iterable[_FutureT[_T]], *, loop: Optional[AbstractEventLoop] = ..., timeout: Optional[float] = ...,
         return_when: str = ...) -> Future[Tuple[Set[Future[_T]], Set[Future[_T]]]]: ...
```

Hints from issue discussion:
FWIW this also fails on master and Python 3.7. I wasn't able to find a simple repro not-involving `async/await`.
Here I have a simple repro:
```
from typing import Union, Generic, TypeVar

T = TypeVar('T')
T_co = TypeVar('T_co', covariant=True)

class Cov(Generic[T_co]): ...
class Inv(Cov[T]): ...

X = Union[Cov[T], Inv[T]]

def f(x: X[T]) -> T: ...
x: Inv[int]
f(x)
```
This test passes on v0.740, but fails on v0.750. The reason is subtle, but ultimately it is related to switch to the new type alias representation. The point is that `expand_type()` (called from `freshen_function_type_vars()`) calls `make_simplified_union()` when visiting union types, while type alias to union stays "unsimplified". Then later constraint inference fails, because it is generally more tricky for union types. I see two possible solutions here:
* Call `make_simplified_union()` in `infer_constraints()` to only use union inference when necessary
* Improve `any_constraints()` to consider cases where two constrain lists are `[T <: X, T :> X]` and `[T :> X]`.

I am leaning towards first option. Also I don't really want to call `get_proper_type()` in `expand_type()` to trigger union simplification, since that looks a bit unprincipled and potentially dangerous.
I agree that using `make_simplified_union` in `infer_constraints()` sounds like the best approach. Type inference shouldn't be affected by whether union types have bee simplified.

Commit message: Fix Python 3.5

Patch to review:
diff --git a/mypy/constraints.py b/mypy/constraints.py
index 9da4f04d2277..fdb1ab0d74d4 100644
--- a/mypy/constraints.py
+++ b/mypy/constraints.py
@@ -108,6 +108,12 @@ def _infer_constraints(template: Type, actual: Type,
     template = get_proper_type(template)
     actual = get_proper_type(actual)
 
+    # Type inference shouldn't be affected by whether union types have bee simplified.
+    if isinstance(template, UnionType):
+        template = mypy.typeops.make_simplified_union(template.items)
+    if isinstance(actual, UnionType):
+        actual = mypy.typeops.make_simplified_union(actual.items)
+
     # Ignore Any types from the type suggestion engine to avoid them
     # causing us to infer Any in situations where a better job could
     # be done otherwise. (This can produce false positives but that
diff --git a/test-data/unit/check-inference.test b/test-data/unit/check-inference.test
index b2050a127ba6..5366de858018 100644
--- a/test-data/unit/check-inference.test
+++ b/test-data/unit/check-inference.test
@@ -2924,3 +2924,18 @@ reveal_type(q2(z))  # N: Revealed type is '__main__.B*'
 reveal_type(q1(Z(b)))  # N: Revealed type is '__main__.B*'
 reveal_type(q2(Z(b)))  # N: Revealed type is '__main__.B*'
 [builtins fixtures/isinstancelist.pyi]
+
+[case testUnionInvariantSubClassAndCovariantBase]
+from typing import Union, Generic, TypeVar
+
+T = TypeVar('T')
+T_co = TypeVar('T_co', covariant=True)
+
+class Cov(Generic[T_co]): ...
+class Inv(Cov[T]): ...
+
+X = Union[Cov[T], Inv[T]]
+
+def f(x: X[T]) -> T: ...
+x: Inv[int]
+reveal_type(f(x))  # N: Revealed type is 'builtins.int*'
diff --git a/test-data/unit/check-overloading.test b/test-data/unit/check-overloading.test
index 6574cd9e730c..0351111c8fa0 100644
--- a/test-data/unit/check-overloading.test
+++ b/test-data/unit/check-overloading.test
@@ -3646,7 +3646,7 @@ def test_narrow_none() -> None:
     a: Optional[int]
     if int():
         a = narrow_none(a)
-        reveal_type(a)  # N: Revealed type is 'Union[builtins.int, None]'
+        reveal_type(a)  # N: Revealed type is 'builtins.int'
 
     b: int
     if int():
@@ -3710,7 +3710,7 @@ def test_narrow_none_v2() -> None:
     a: Optional[int]
     if int():
         a = narrow_none_v2(a)
-        reveal_type(a)  # N: Revealed type is 'Union[builtins.int, None]'
+        reveal_type(a)  # N: Revealed type is 'builtins.int'
 
     b: int
     if int():
diff --git a/test-data/unit/pythoneval.test b/test-data/unit/pythoneval.test
index fad202d50d1e..d6df7ece9899 100644
--- a/test-data/unit/pythoneval.test
+++ b/test-data/unit/pythoneval.test
@@ -1490,3 +1490,12 @@ reveal_type(A().b)
 [out]
 _testNamedTupleAtRunTime.py:5: error: NamedTuple type as an attribute is not supported
 _testNamedTupleAtRunTime.py:7: note: Revealed type is 'Any'
+
+[case testAsyncioFutureWait]
+# mypy: strict-optional
+from asyncio import Future, wait
+from typing import List
+
+async def foo() -> None:
+    f = []  # type: List[Future[None]]
+    await wait(f)


Code review comments:
diff_hunk: @@ -108,6 +108,12 @@ def _infer_constraints(template: Type, actual: Type,
     template = get_proper_type(template)
     actual = get_proper_type(actual)
 
+    # Type inference shouldn't be affected by whether union types have bee simplified.
path: mypy/constraints.py
text: @user1:
typo: `been`