Repository: python/mypy
Pull request: Add symbols for member variables to the symbol table when a subclass of TypedDict with the same name exists
PR title a little long - suggestions welcome.

Fixes #8097 

Problem statement:
Forward reference to namespaced type with same name breaks with TypedDict
* Are you reporting a bug, or opening a feature request?

I am reporting a bug

* Please insert below the code you are checking with mypy,
  or a mock-up repro if the source is private. We would appreciate
  if you try to simplify your case to a minimal repro.

```
from __future__ import annotations
from typing import Union, Any
from typing_extensions import TypedDict


class T(TypedDict):
    b: b.T


class b:
    T = Union[Any]
```

* What is the actual behavior/output?

```
failure2.py:7: error: Name 'b.T' is not defined
Found 1 error in 1 file (checked 1 source file)
```
* What is the behavior/output you expect? 

No type error

* What are the versions of mypy and Python you are using?
  Do you see the same issue after installing mypy from Git master?

Python 3.7.3, mypy 0.750. I get the same results after installing mypy from Git master.

* What are the mypy flags you are using? (For example --strict-optional)

No particular flag.

* If mypy crashed with a traceback, please paste
  the full traceback below.

No crash

Additional notes:
The following examples *do* typecheck properly:
* Name the type differently:
```
from __future__ import annotations
from typing import Union, Any
from typing_extensions import TypedDict


class T(TypedDict):
    b: b.U


class b:
    U = Union[Any]
```
* Without TypedDict:
```
from __future__ import annotations
from typing import Union, Any


class T:
    b: b.T


class b:
    T = Union[Any]
```
* Without forward reference:
```
from __future__ import annotations
from typing import Union, Any
from typing_extensions import TypedDict


class b:
    T = Union[Any]


class T(TypedDict):
    b: b.T
```

Hints from issue discussion:
Thanks for the detailed bug report! Something is clearly wrong here.
Well, I encounter a similar situation with class T inherited from `NamedTuple`. It seems like this kind of inheritance cause TypeInfo of class `b` to have no node of attribute `T`, but I'm still trying to figure out the deeper reason.
Hey! I had a look into this one to see if I could find the reason for you. I've had a go at trying to explain what I found, and have opened a PR so you can see a potential fix. 

When the first class is analysed we add it to missing_names. This is because the first analysis of the TypeDict will be incomplete so we don't want to add its type information anywhere. 

But then when analysing the second class we then don't add the member T's symbol to the symbol table because we see it under the missing_names variable.

This behaviour was added in https://github.com/python/mypy/pull/6257 and is expected to apply to anything that defines a name in the current scope. But in the example given shows that in the case of TypedDicts we are ignoring not just at this top scope. 

My initial thought was to update add_symbol_table_node() in seminal.py to add to the symbol table when the symbol has kind MDEF (when it's a member variable). This solved this example, but any feedback/other edge cases would be great! 

This works because we want the class T to be ignored (because it's a subclass of TypedDict), but we still want the symbol for member T to be added to the symbol table. Once added to the symbol table it can then be found.

Commit message: Separate missing_names based on scope

Patch to review:
diff --git a/mypy/semanal.py b/mypy/semanal.py
index 42770e1a0389..813834a935b1 100644
--- a/mypy/semanal.py
+++ b/mypy/semanal.py
@@ -49,11 +49,12 @@
 """
 
 from contextlib import contextmanager
+from collections import defaultdict
 
 from typing import (
     List, Dict, Set, Tuple, cast, TypeVar, Union, Optional, Callable, Iterator, Iterable
 )
-from typing_extensions import Final
+from typing_extensions import DefaultDict, Final
 
 from mypy.nodes import (
     MypyFile, TypeInfo, Node, AssignmentStmt, FuncDef, OverloadedFuncDef,
@@ -199,7 +200,7 @@ class SemanticAnalyzer(NodeVisitor[None],
     #
     # Note that a star import adds a special name '*' to the set, this blocks
     # adding _any_ names in the current file.
-    missing_names = None  # type: Set[str]
+    missing_names = None  # type: DefaultDict[Optional[str], Set[str]]
     # Callbacks that will be called after semantic analysis to tweak things.
     patches = None  # type: List[Tuple[int, Callable[[], None]]]
     loop_depth = 0         # Depth of breakable loops
@@ -356,7 +357,7 @@ def refresh_partial(self,
         self.deferred = False
         self.incomplete = False
         self._final_iteration = final_iteration
-        self.missing_names = set()
+        self.missing_names = defaultdict(set)
 
         with self.file_context(file_node, options, active_type):
             if isinstance(node, MypyFile):
@@ -4170,6 +4171,12 @@ def lookup_current_scope(self, name: str) -> Optional[SymbolTableNode]:
         else:
             return self.globals.get(name)
 
+    def current_scope_identifier(self) -> Optional[str]:
+        if self.type and self.is_class_scope():
+            return self.type._fullname
+        else:
+            return self.cur_mod_id
+
     #
     # Adding symbols
     #
@@ -4273,7 +4280,8 @@ def add_symbol_table_node(self,
                 if not (isinstance(new, (FuncDef, Decorator))
                         and self.set_original_def(old, new)):
                     self.name_already_defined(name, context, existing)
-        elif name not in self.missing_names and '*' not in self.missing_names:
+        elif (name not in self.missing_names[self.current_scope_identifier()]
+                and '*' not in self.missing_names[self.current_scope_identifier()]):
             names[name] = symbol
             self.progress = True
             return True
@@ -4445,7 +4453,7 @@ def mark_incomplete(self, name: str, node: Node,
             placeholder = PlaceholderNode(fullname, node, self.statement.line,
                                           becomes_typeinfo=becomes_typeinfo)
             self.add_symbol(name, placeholder, context=dummy_context())
-        self.missing_names.add(name)
+        self.missing_names[self.current_scope_identifier()].add(name)
 
     def is_incomplete_namespace(self, fullname: str) -> bool:
         """Is a module or class namespace potentially missing some definitions?
diff --git a/test-data/unit/check-newsemanal.test b/test-data/unit/check-newsemanal.test
index 58153c58c37d..e77e05277729 100644
--- a/test-data/unit/check-newsemanal.test
+++ b/test-data/unit/check-newsemanal.test
@@ -3200,3 +3200,25 @@ class User:
     def __init__(self, name: str) -> None:
         self.name = name  # E: Cannot assign to a method \
                           # E: Incompatible types in assignment (expression has type "str", variable has type "Callable[..., Any]")
+
+[case testNewAnalyzerMemberNameMatchesTypedDict]
+from typing import Union, Any
+from typing_extensions import TypedDict
+
+class T(TypedDict):
+    b: b.T
+
+class b:
+    T: Union[Any]
+[builtins fixtures/tuple.pyi]
+
+[case testNewAnalyzerMemberNameMatchesNamedTuple]
+from typing import Union, Any, NamedTuple
+
+class T(NamedTuple):
+    b: b.T
+
+class b:
+    T = Union[Any]
+
+[builtins fixtures/tuple.pyi]


Code review comments:
diff_hunk: @@ -4170,6 +4171,12 @@ def lookup_current_scope(self, name: str) -> Optional[SymbolTableNode]:
         else:
             return self.globals.get(name)
 
+    def current_scope_identifier(self) -> Optional[str]:
path: mypy/semanal.py
text: @user1:
This is a little inconsistent with how we track scopes elsewhere. It would be better to mirror the logic used for the `locals` attribute (also `block_depth` a few others). We'd maintain a stack and append a fresh set on each `enter()` or `enter_class()` call, and pop an item on each `leave()` or `leave_class()` call. We can then use `missing_names[-1]` to access the innermost scope.

@author:
That makes a lot more sense, cheers for the hint!