Repository: python/mypy

Issue:
mypy returns `invalid type: "bytes"` in classes that define a `bytes` instance attribute
When trying to add type annotations to a project (https://github.com/DCSO/flor), `mypy` would complain that `bytes` was not a valid type, which was quite puzzling to say the least. After some head scratching and searching in the issue log I was able to reproduce the problem with a simple module (Python 3.5.2+ with mypy 0.511):

```
class Foo(object):

    def __init__(self) -> None:
        self.bytes = bytearray(b"foo")

    def bar(self, f : bytes):
        return f + b"test"

foo = Foo()

foo.bar(b"test")
```

Running `mypy example.py` returns the following error messages:

```
example.py:6: error: Invalid type "bytes"
example.py:7: error: Unsupported left operand type for + (object)
```

When renaming `self.bytes` to `self._bytes`, no issues are reported. It seems that mypy mistakes `self.bytes` for a class variable, although it is an instance variable and not visible at the declaration time of function `bar`.


Discussion:
Thanks for reporting this! I reproduced the issue on 0.511, what's more concerning is that this is a crash on master!

```
test.py:6: error: INTERNAL ERROR -- please report a bug at https://github.com/python/mypy/issues version: 0.520-dev-005fd6b357b995f13e3d90293cabf3c8c8c2a952
Traceback (most recent call last):
  File "C:\Python36\Scripts\mypy-script.py", line 11, in <module>
    load_entry_point('mypy', 'console_scripts', 'mypy')()
  File "c:\users\ethanhs\documents\mypy\mypy\__main__.py", line 7, in console_entry
    main(None)
  File "c:\users\ethanhs\documents\mypy\mypy\main.py", line 50, in main
    res = type_check_only(sources, bin_dir, options)
  File "c:\users\ethanhs\documents\mypy\mypy\main.py", line 97, in type_check_only
    options=options)
  File "c:\users\ethanhs\documents\mypy\mypy\build.py", line 196, in build
    graph = dispatch(sources, manager)
  File "c:\users\ethanhs\documents\mypy\mypy\build.py", line 1769, in dispatch
    process_graph(graph, manager)
  File "c:\users\ethanhs\documents\mypy\mypy\build.py", line 2012, in process_graph
    process_stale_scc(graph, scc, manager)
  File "c:\users\ethanhs\documents\mypy\mypy\build.py", line 2107, in process_stale_scc
    graph[id].semantic_analysis()
  File "c:\users\ethanhs\documents\mypy\mypy\build.py", line 1664, in semantic_analysis
    self.manager.semantic_analyzer.visit_file(self.tree, self.xpath, self.options, patches)
  File "c:\users\ethanhs\documents\mypy\mypy\semanal.py", line 295, in visit_file
    self.accept(d)
  File "c:\users\ethanhs\documents\mypy\mypy\semanal.py", line 3636, in accept
    node.accept(self)
  File "c:\users\ethanhs\documents\mypy\mypy\nodes.py", line 754, in accept
    return visitor.visit_class_def(self)
  File "c:\users\ethanhs\documents\mypy\mypy\semanal.py", line 668, in visit_class_def
    defn.defs.accept(self)
  File "c:\users\ethanhs\documents\mypy\mypy\nodes.py", line 815, in accept
    return visitor.visit_block(self)
  File "c:\users\ethanhs\documents\mypy\mypy\semanal.py", line 1522, in visit_block
    self.accept(s)
  File "c:\users\ethanhs\documents\mypy\mypy\semanal.py", line 3636, in accept
    node.accept(self)
  File "c:\users\ethanhs\documents\mypy\mypy\nodes.py", line 565, in accept
    return visitor.visit_func_def(self)
  File "c:\users\ethanhs\documents\mypy\mypy\semanal.py", line 418, in visit_func_def
    self.analyze_function(defn)
  File "c:\users\ethanhs\documents\mypy\mypy\semanal.py", line 598, in analyze_function
    defn.type = self.type_analyzer().visit_callable_type(defn.type, nested=False)
  File "c:\users\ethanhs\documents\mypy\mypy\typeanal.py", line 331, in visit_callable_type
    ret = t.copy_modified(arg_types=self.anal_array(t.arg_types, nested=nested),
  File "c:\users\ethanhs\documents\mypy\mypy\typeanal.py", line 520, in anal_array
    res.append(self.anal_type(t, nested))
  File "c:\users\ethanhs\documents\mypy\mypy\typeanal.py", line 527, in anal_type
    return t.accept(self)
  File "c:\users\ethanhs\documents\mypy\mypy\types.py", line 189, in accept
    return visitor.visit_unbound_type(self)
  File "c:\users\ethanhs\documents\mypy\mypy\typeanal.py", line 166, in visit_unbound_type
    tvar_def = self.tvar_scope.get_binding(sym)
  File "c:\users\ethanhs\documents\mypy\mypy\tvar_scope.py", line 71, in get_binding
    assert fullname is not None
AssertionError:
test.py:6: note: use --pdb to drop into pdb
```
As expected, renaming `self.bytes` resolved it.
The crash should be not hard to fix.
@JukkaL If necessary, I can take a look at it right now, otherwise I would prefer to wait until after the release.
Let's wait until the release since this also happens on 0.511.
Ah so it's an error on 0.511 but a crash on master? Hmm I still want to move forward with the release, but if the fix is simple we may release 0.521 later this week.
There are probably a few more candidates for 0.521...

On Mon, Jul 10, 2017 at 2:02 PM, Jukka Lehtosalo <notifications@github.com>
wrote:

> Ah so it's an error on 0.511 but a crash on master? Hmm I still want to
> move forward with the release, but if the fix is simple we may release
> 0.521 later this week.
>
> —
> You are receiving this because you are subscribed to this thread.
> Reply to this email directly, view it on GitHub
> <https://github.com/python/mypy/issues/3687#issuecomment-314241297>, or mute
> the thread
> <https://github.com/notifications/unsubscribe-auth/ACwrMnSDeVIoFyLMlHa435jsra4ctQ9tks5sMpFxgaJpZM4OTT-K>
> .
>



-- 
--Guido van Rossum (python.org/~guido)

Here's a simpler example that generates the crash:

```py
class C:
    def __init__(self) -> None:
        self.int = ''

    def f(self, f: int) -> None:
        f.x
```

Since the original issue hasn't been reported previously, this may be a fairly rare thing. The original error also was tricky to `# type: ignore` so likely there isn't a lot of code that has the error ignored (which likely would still cause a crash, however).

Resolving patch:
diff --git a/mypy/nodes.py b/mypy/nodes.py
--- a/mypy/nodes.py
+++ b/mypy/nodes.py
@@ -2279,11 +2279,14 @@ class SymbolTableNode:
     cross_ref = None  # type: Optional[str]
     # Was this node created by normalіze_type_alias?
     normalized = False  # type: bool
+    # Was this defined by assignment to self attribute?
+    implicit = False  # type: bool
 
     def __init__(self, kind: int, node: Optional[SymbolNode], mod_id: str = None,
                  typ: 'mypy.types.Type' = None,
                  module_public: bool = True, normalized: bool = False,
-                 alias_tvars: Optional[List[str]] = None) -> None:
+                 alias_tvars: Optional[List[str]] = None,
+                 implicit: bool = False) -> None:
         self.kind = kind
         self.node = node
         self.type_override = typ
@@ -2291,6 +2294,7 @@ def __init__(self, kind: int, node: Optional[SymbolNode], mod_id: str = None,
         self.module_public = module_public
         self.normalized = normalized
         self.alias_tvars = alias_tvars
+        self.implicit = implicit
 
     @property
     def fullname(self) -> Optional[str]:
@@ -2334,6 +2338,10 @@ def serialize(self, prefix: str, name: str) -> JsonDict:
                 }  # type: JsonDict
         if not self.module_public:
             data['module_public'] = False
+        if self.normalized:
+            data['normalized'] = True
+        if self.implicit:
+            data['implicit'] = True
         if self.kind == MODULE_REF:
             assert self.node is not None, "Missing module cross ref in %s for %s" % (prefix, name)
             data['cross_ref'] = self.node.fullname()
@@ -2371,6 +2379,10 @@ def deserialize(cls, data: JsonDict) -> 'SymbolTableNode':
                 stnode.alias_tvars = data['alias_tvars']
         if 'module_public' in data:
             stnode.module_public = data['module_public']
+        if 'normalized' in data:
+            stnode.normalized = data['normalized']
+        if 'implicit' in data:
+            stnode.implicit = data['implicit']
         return stnode
 
 
diff --git a/mypy/semanal.py b/mypy/semanal.py
--- a/mypy/semanal.py
+++ b/mypy/semanal.py
@@ -1809,11 +1809,12 @@ def analyze_member_lvalue(self, lval: MemberExpr) -> None:
             lval.is_def = True
             v = Var(lval.name)
             v.set_line(lval)
+            v._fullname = self.qualified_name(lval.name)
             v.info = self.type
             v.is_ready = False
             lval.def_var = v
             lval.node = v
-            self.type.names[lval.name] = SymbolTableNode(MDEF, v)
+            self.type.names[lval.name] = SymbolTableNode(MDEF, v, implicit=True)
         self.check_lvalue_validity(lval.node, lval)
 
     def is_self_member_ref(self, memberexpr: MemberExpr) -> bool:
@@ -3400,7 +3401,9 @@ def lookup(self, name: str, ctx: Context) -> SymbolTableNode:
                 return None
         # 2. Class attributes (if within class definition)
         if self.is_class_scope() and name in self.type.names:
-            return self.type.names[name]
+            node = self.type.names[name]
+            if not node.implicit:
+                return node
         # 3. Local (function) scopes
         for table in reversed(self.locals):
             if table is not None and name in table:


Test patch:
diff --git a/test-data/unit/check-incremental.test b/test-data/unit/check-incremental.test
--- a/test-data/unit/check-incremental.test
+++ b/test-data/unit/check-incremental.test
@@ -2735,3 +2735,85 @@ class A:
         pass
 a = A()
 [builtins fixtures/property.pyi]
+
+[case testClassNamesResolutionCrashAccess]
+import mod
+
+[file mod.py]
+class C:
+    def __init__(self) -> None:
+        self.int = ''
+
+    def f(self, f: int) -> None:
+        pass
+
+[file mod.py.2]
+class C:
+    def __init__(self) -> None:
+        self.int = ''
+
+    def f(self, f: int) -> None:
+        f.x
+
+[out]
+[out2]
+tmp/mod.py:6: error: "int" has no attribute "x"
+
+[case testClassNamesResolutionCrashReadCache]
+import mod
+
+[file mod.py]
+import submod
+
+[file mod.py.2]
+from submod import C
+
+c = C()
+reveal_type(c.int)
+reveal_type(c.y)
+
+[file submod.py]
+from typing import List
+
+class C:
+    def __init__(self) -> None:
+        self.int = []  # type: List[int]
+
+    def f(self, f: int) -> None:
+        self.y = f
+
+[builtins fixtures/list.pyi]
+[out]
+[out2]
+tmp/mod.py:4: error: Revealed type is 'builtins.list[builtins.int]'
+tmp/mod.py:5: error: Revealed type is 'builtins.int'
+
+[case testClassNamesResolutionCrashReveal]
+import mod
+
+[file mod.py]
+class Foo(object):
+
+    def __init__(self) -> None:
+        self.bytes = b"foo"
+
+    def bar(self, f: bytes):
+        pass
+
+foo = Foo()
+foo.bar(b"test")
+
+[file mod.py.2]
+class Foo(object):
+
+    def __init__(self) -> None:
+        self.bytes = b"foo"
+
+    def bar(self, f: bytes):
+        reveal_type(f)
+
+foo = Foo()
+foo.bar(b"test")
+[out]
+[out2]
+tmp/mod.py:7: error: Revealed type is 'builtins.bytes'