Repository: python/mypy

Issue:
Unexpected type 'unicode' for os.path.join() with (Any, str) args
The revealed type for the `os.path.join` call in the program below is `unicode`, even though I'd expect either `Any` or `str`, as the arguments have types `Any` and `str`:

```py
import os
def f(): pass
reveal_type(os.path.join(f(), 'x'))  # builtins.unicode
```

Here's a self-contained example:

```py
# a.pyi
from typing import overload
@overload
def join(__p1: bytes, *p: bytes) -> bytes: ...
@overload
def join(__p1: unicode, *p: unicode) -> unicode: ...
```

```py
# program.py
from a import join
def f(): pass
reveal_type(join(f(), 'x'))  # builtins.unicode
```

Run as `mypy -2 program.py`.

@Michael0x2a Do you have time to have a look at this? This is causing issues in internal Dropbox repos. If you are busy, I can try to fix this. This was apparently introduced by f61c2baac5915ab680c2f1a9858b7f608262a00e.


Discussion:
I am looking at this now.
I think the issue is that due to the way we handle promotions in mypy, `bytes` is actually technically considered to be a subtype of `unicode` -- so the union math logic concludes that it's valid to return `unicode` instead of `Any` since the former is more precise.

Not sure what the best way to handle this is -- I'd personally prefer just removing that promotion altogether, but that's a very invasive and difficult-to-implement change. Alternatively, we could special-case this in the union math somehow, but that would make it inconsistent with the rest of mypy. E.g. this currently typechecks, for example:

    def expects_unicode(x):
        # type: (unicode) -> None
        pass

    b = b'foo'  # type: bytes

    expects_unicode(b)

I guess other options include:

1. Modifying the "is the last return type a supertype of the previous ones" checks so promotions are disabled altogether.
2. Removing that check altogether so we always return Any.
My original motivation for requesting it was that this example should work
```python
@overload
def f(x: List[int]) -> List[int]: ...
@overload
def f(x: List[Any]) -> List[Any]: ...

arg: Any
f(arg)  # should be List[Any], _not_ just Any
```
Also now I think that we should really check both ways, because the logic is that replacing a precise type with `Any` should ideally not cause _new_ errors. For example:
```python
def f() -> int: ...

@overload
def g(x: int) -> int: ...
@overload
def g(x: float) -> float: ...

x: int = g(f())  # this passes, but will fail if I remove the annotation from 'f'
```

I will now prepare a PR to return last type only if it is compatible both ways with all other return types (like in the first example with `List[Any]`).

Resolving patch:
diff --git a/mypy/checkexpr.py b/mypy/checkexpr.py
--- a/mypy/checkexpr.py
+++ b/mypy/checkexpr.py
@@ -1304,8 +1304,10 @@ def infer_overload_return_type(self,
             return None
         elif any_causes_overload_ambiguity(matches, return_types, arg_types, arg_kinds, arg_names):
             # An argument of type or containing the type 'Any' caused ambiguity.
-            if all(is_subtype(ret_type, return_types[-1]) for ret_type in return_types[:-1]):
-                # The last match is a supertype of all the previous ones, so it's safe
+            if all(is_subtype(ret_type, return_types[-1]) and
+                   is_subtype(return_types[-1], ret_type)
+                   for ret_type in return_types[:-1]):
+                # The last match is mutually compatible with all previous ones, so it's safe
                 # to return that inferred type.
                 return return_types[-1], inferred_types[-1]
             else:


Test patch:
diff --git a/test-data/unit/check-overloading.test b/test-data/unit/check-overloading.test
--- a/test-data/unit/check-overloading.test
+++ b/test-data/unit/check-overloading.test
@@ -1276,7 +1276,7 @@ def f(x: object) -> object: ...
 def f(x): pass
 
 a: Any
-reveal_type(f(a))  # E: Revealed type is 'builtins.object'
+reveal_type(f(a))  # E: Revealed type is 'Any'
 
 [case testOverloadWithOverlappingItemsAndAnyArgument2]
 from typing import overload, Any
@@ -1288,7 +1288,7 @@ def f(x: float) -> float: ...
 def f(x): pass
 
 a: Any
-reveal_type(f(a))  # E: Revealed type is 'builtins.float'
+reveal_type(f(a))  # E: Revealed type is 'Any'
 
 [case testOverloadWithOverlappingItemsAndAnyArgument3]
 from typing import overload, Any
@@ -1312,17 +1312,16 @@ def f(x: object, y: int, z: str) -> object: ...
 def f(x): pass
 
 a: Any
-# Any causes ambiguity; we fall back to returning object since it's a
-# supertype of int
-reveal_type(f(a, 1, ''))  # E: Revealed type is 'builtins.object'
+# Any causes ambiguity
+reveal_type(f(a, 1, ''))  # E: Revealed type is 'Any'
 # Any causes no ambiguity
 reveal_type(f(1, a, a))  # E: Revealed type is 'builtins.int'
 reveal_type(f('', a, a))  # E: Revealed type is 'builtins.object'
 # Like above, but use keyword arguments.
-reveal_type(f(y=1, z='', x=a))  # E: Revealed type is 'builtins.object'
+reveal_type(f(y=1, z='', x=a))  # E: Revealed type is 'Any'
 reveal_type(f(y=a, z='', x=1))  # E: Revealed type is 'builtins.int'
 reveal_type(f(z='', x=1, y=a))  # E: Revealed type is 'builtins.int'
-reveal_type(f(z='', x=a, y=1))  # E: Revealed type is 'builtins.object'
+reveal_type(f(z='', x=a, y=1))  # E: Revealed type is 'Any'
 
 [case testOverloadWithOverlappingItemsAndAnyArgument5]
 from typing import overload, Any, Union
@@ -1334,7 +1333,7 @@ def f(x: Union[int, float]) -> float: ...
 def f(x): pass
 
 a: Any
-reveal_type(f(a))  # E: Revealed type is 'builtins.float'
+reveal_type(f(a))  # E: Revealed type is 'Any'
 
 [case testOverloadWithOverlappingItemsAndAnyArgument6]
 from typing import overload, Any
@@ -1372,7 +1371,7 @@ def g(x): pass
 
 a: Any
 reveal_type(f(1, *a))  # E: Revealed type is 'builtins.int'
-reveal_type(g(1, *a))  # E: Revealed type is 'builtins.object'
+reveal_type(g(1, *a))  # E: Revealed type is 'Any'
 
 [case testOverloadWithOverlappingItemsAndAnyArgument8]
 from typing import overload, Any
diff --git a/test-data/unit/python2eval.test b/test-data/unit/python2eval.test
--- a/test-data/unit/python2eval.test
+++ b/test-data/unit/python2eval.test
@@ -342,7 +342,7 @@ print b.x + 1
 [out]
 4
 
-[case testReModuleBytesPython2]
+[case testReModuleBytes_python2]
 # Regression tests for various overloads in the re module -- bytes version
 import re
 if False:
@@ -365,7 +365,7 @@ if False:
     re.subn(bpat, lambda m: b'', b'')[0] + b''
 [out]
 
-[case testReModuleStringPython2]
+[case testReModuleString_python2]
 # Regression tests for various overloads in the re module -- string version
 import re
 ure = u'a+'
@@ -393,3 +393,11 @@ def g():  # type: () -> int
     yield
 [out]
 _program.py:2: error: The return type of a generator function should be "Generator" or one of its supertypes
+
+[case testOsPathJoinWorksWithAny_python2]
+import os
+def f():  # no annotation
+   return 'tests'
+path = 'test'
+path = os.path.join(f(), 'test.py')
+[out]
diff --git a/test-data/unit/pythoneval.test b/test-data/unit/pythoneval.test
--- a/test-data/unit/pythoneval.test
+++ b/test-data/unit/pythoneval.test
@@ -1109,9 +1109,11 @@ async def main() -> None:
     ((a_x, a_y),) = await asyncio.gather(get_location('start'))
     reveal_type(a_x)
     reveal_type(a_y)
+reveal_type(asyncio.gather(*[asyncio.sleep(1), asyncio.sleep(1)]))
 [out]
 _testAsyncioGatherPreciseType.py:9: error: Revealed type is 'builtins.str'
 _testAsyncioGatherPreciseType.py:10: error: Revealed type is 'builtins.str'
+_testAsyncioGatherPreciseType.py:11: error: Revealed type is 'asyncio.futures.Future[builtins.tuple[Any]]'
 
 [case testMultipleInheritanceWorksWithTupleTypeGeneric]
 from typing import SupportsAbs, NamedTuple