Repository: python/mypy

Issue:
Python version-conditional import inside function does not respect test
This is OK at the top level:

``` python
import sys
if sys.version_info[0] >= 3:
    import configparser
else:
    import ConfigParser as configparser
```

This causes the error as noted in the comment:

``` python
import sys
def hooper(): # type: () -> None
    if sys.version_info[0] >= 3:
        import configparser # if mypy -2: Cannot find module named 'configparser'
    else:
        import ConfigParser as configparser # if mypy: Cannot find module named 'ConfigParser'
```



Discussion:
Interesting. It looks like other cases of sys.version_info checks (not involving import) work correctly, so it seems the sys.version_info logic is executed here (also verified in the debugger). So maybe the code for understanding alternative imports isn't triggered inside functions?

> So maybe the code for understanding alternative imports isn't triggered inside functions?

Even `typing.TYPE_CHECKING` seems to be ignored inside a function:

``` python
#!/usr/bin/env python

from __future__ import print_function
from typing import TYPE_CHECKING
import sys

def coolfun(): # type: () -> None
    if sys.version_info[0] >= 3:
        print("i'm py3 in a function")
    else:
        if not TYPE_CHECKING:
            import ConfigParser as configparser # hooper.py:12: error: Cannot find module named 'ConfigParser'
        print("i'm py2 in a function")

if __name__ == '__main__':
    coolfun()
```

Yeah, it seems that inside functions the is_unreachable flag is not
respected by the import-handling code. I'll have to delve in deeper to
understand why that is -- if you replace the alternative imports with
conflicting assignments you'll see that the type checker does honor
reachability everywhere. And the import code respects it outside functions.

I debugged this a bit more, it's because FirstPass doesn't descend into
functions. I don't think this is a very important use case -- it should be
easy to work around by moving the conditional imports out of functions.

It's quite common for imports, conditional or otherwise, to be done within functions (to improve startup time, to not bloat the interpreter's footprint unless a module is needed, to prevent namespace pollution, etc.). It's also fairly surprising for it to not work as expected.

Do you feel up to submitting a PR? It's not a particularly complex part of
mypy.

I've never looked at mypy's internals apart from some collections stuff, so if you already have an idea of how to fix it, that's less likely to break something than if I have to figure it out. If you're busy, I don't mind taking a look, but unless the fix is obvious, you could probably take care of it faster than I could.

But probably not sooner. :-) We have over 450 issues in the tracker and we
have to prioritize.

Any tips on what needs to be changed and how? (I guess the where is `FirstPass` of `mypy/semanal.py`.)

I'm guessing the function and class visitors need to recurse into their
bodies. And we'll need tests. (Does the same problem occur with conditional
imports in classes? If not the class visitor might not need fixing. The
visitors for some other statements show you how to recurse.)

To traverse function and class bodies, I see these (from `NodeVisitor`) are already present:

``` python
def visit_block(self, o: 'mypy.nodes.Block') -> T:
def visit_while_stmt(self, o: 'mypy.nodes.WhileStmt') -> T:
def visit_for_stmt(self, o: 'mypy.nodes.ForStmt') -> T:
def visit_if_stmt(self, o: 'mypy.nodes.IfStmt') -> T:
def visit_try_stmt(self, o: 'mypy.nodes.TryStmt') -> T:
def visit_with_stmt(self, o: 'mypy.nodes.WithStmt') -> T:
```

So I guess only the function and class visitors should need changes, and the import-related stuff as to whether it's a top-level import or not (maybe?)

I expect the import visitors to be fine, it's the function/class visitors
and the statement visitors that you noticed that need work. (visitors
containing "pass" are _not_ sufficient, they have to do the work
themselves. There are plenty of examples.)

On Sun, Oct 16, 2016 at 4:51 PM, Mark Laws notifications@github.com wrote:

> To traverse function and class bodies, I see these (from NodeVisitor) are
> already present
> 
> def visit_block(self, o: 'mypy.nodes.Block') -> T: passdef visit_while_stmt(self, o: 'mypy.nodes.WhileStmt') -> T: passdef visit_for_stmt(self, o: 'mypy.nodes.ForStmt') -> T: passdef visit_if_stmt(self, o: 'mypy.nodes.IfStmt') -> T: passdef visit_try_stmt(self, o: 'mypy.nodes.TryStmt') -> T: passdef visit_with_stmt(self, o: 'mypy.nodes.WithStmt') -> T: pass
> 
> So I guess only the function and class visitors should need changes, and
> the import-related stuff as to whether it's a top-level import or not
> (maybe?)
> 
> —
> You are receiving this because you commented.
> Reply to this email directly, view it on GitHub
> https://github.com/python/mypy/issues/2260#issuecomment-254086105, or mute
> the thread
> https://github.com/notifications/unsubscribe-auth/ACwrMnz69kzUPt1KcpaO-vMvBEgtl81Lks5q0riQgaJpZM4KYAxm
> .

## 

--Guido van Rossum (python.org/~guido)

Are you still working on a PR for this?

Yes, sorry--I got sidetracked but I should have some time to look at it today.


Resolving patch:
diff --git a/mypy/semanal.py b/mypy/semanal.py
--- a/mypy/semanal.py
+++ b/mypy/semanal.py
@@ -2557,6 +2557,9 @@ def is_func_scope(self) -> bool:
     def is_class_scope(self) -> bool:
         return self.type is not None and not self.is_func_scope()
 
+    def is_module_scope(self) -> bool:
+        return not (self.is_class_scope() or self.is_func_scope())
+
     def add_symbol(self, name: str, node: SymbolTableNode,
                    context: Context) -> None:
         if self.is_func_scope():
@@ -2739,14 +2742,16 @@ def visit_block(self, b: Block) -> None:
         self.sem.block_depth[-1] -= 1
 
     def visit_assignment_stmt(self, s: AssignmentStmt) -> None:
-        for lval in s.lvalues:
-            self.analyze_lvalue(lval, explicit_type=s.type is not None)
+        if self.sem.is_module_scope():
+            for lval in s.lvalues:
+                self.analyze_lvalue(lval, explicit_type=s.type is not None)
 
     def visit_func_def(self, func: FuncDef) -> None:
         sem = self.sem
         func.is_conditional = sem.block_depth[-1] > 0
         func._fullname = sem.qualified_name(func.name())
-        if func.name() in sem.globals:
+        at_module = sem.is_module_scope()
+        if at_module and func.name() in sem.globals:
             # Already defined in this module.
             original_sym = sem.globals[func.name()]
             if original_sym.kind == UNBOUND_IMPORTED:
@@ -2761,25 +2766,41 @@ def visit_func_def(self, func: FuncDef) -> None:
                 # Report error.
                 sem.check_no_global(func.name(), func)
         else:
-            sem.globals[func.name()] = SymbolTableNode(GDEF, func, sem.cur_mod_id)
+            if at_module:
+                sem.globals[func.name()] = SymbolTableNode(GDEF, func, sem.cur_mod_id)
+            # Also analyze the function body (in case there are conditional imports).
+            sem.function_stack.append(func)
+            sem.errors.push_function(func.name())
+            sem.enter()
+            func.body.accept(self)
+            sem.leave()
+            sem.errors.pop_function()
+            sem.function_stack.pop()
 
     def visit_overloaded_func_def(self, func: OverloadedFuncDef) -> None:
-        self.sem.check_no_global(func.name(), func, True)
+        kind = self.kind_by_scope()
+        if kind == GDEF:
+            self.sem.check_no_global(func.name(), func, True)
         func._fullname = self.sem.qualified_name(func.name())
-        self.sem.globals[func.name()] = SymbolTableNode(GDEF, func,
-                                                        self.sem.cur_mod_id)
+        if kind == GDEF:
+            self.sem.globals[func.name()] = SymbolTableNode(kind, func, self.sem.cur_mod_id)
 
     def visit_class_def(self, cdef: ClassDef) -> None:
-        self.sem.check_no_global(cdef.name, cdef)
+        kind = self.kind_by_scope()
+        if kind == LDEF:
+            return
+        elif kind == GDEF:
+            self.sem.check_no_global(cdef.name, cdef)
         cdef.fullname = self.sem.qualified_name(cdef.name)
         info = TypeInfo(SymbolTable(), cdef, self.sem.cur_mod_id)
         info.set_line(cdef.line, cdef.column)
         cdef.info = info
-        self.sem.globals[cdef.name] = SymbolTableNode(GDEF, info,
-                                                      self.sem.cur_mod_id)
+        if kind == GDEF:
+            self.sem.globals[cdef.name] = SymbolTableNode(kind, info, self.sem.cur_mod_id)
         self.process_nested_classes(cdef)
 
     def process_nested_classes(self, outer_def: ClassDef) -> None:
+        self.sem.enter_class(outer_def)
         for node in outer_def.defs.body:
             if isinstance(node, ClassDef):
                 node.info = TypeInfo(SymbolTable(), node, self.sem.cur_mod_id)
@@ -2791,20 +2812,28 @@ def process_nested_classes(self, outer_def: ClassDef) -> None:
                 symbol = SymbolTableNode(MDEF, node.info)
                 outer_def.info.names[node.name] = symbol
                 self.process_nested_classes(node)
+            elif isinstance(node, (ImportFrom, Import, ImportAll, IfStmt)):
+                node.accept(self)
+        self.sem.leave_class()
 
     def visit_import_from(self, node: ImportFrom) -> None:
         # We can't bind module names during the first pass, as the target module might be
         # unprocessed. However, we add dummy unbound imported names to the symbol table so
         # that we at least know that the name refers to a module.
-        node.is_top_level = True
+        at_module = self.sem.is_module_scope()
+        node.is_top_level = at_module
+        if not at_module:
+            return
         for name, as_name in node.names:
             imported_name = as_name or name
             if imported_name not in self.sem.globals:
                 self.sem.add_symbol(imported_name, SymbolTableNode(UNBOUND_IMPORTED, None), node)
 
     def visit_import(self, node: Import) -> None:
-        node.is_top_level = True
+        node.is_top_level = self.sem.is_module_scope()
         # This is similar to visit_import_from -- see the comment there.
+        if not self.sem.is_module_scope():
+            return
         for id, as_id in node.ids:
             imported_id = as_id or id
             if imported_id not in self.sem.globals:
@@ -2814,28 +2843,31 @@ def visit_import(self, node: Import) -> None:
                 self.sem.globals[imported_id] = SymbolTableNode(UNBOUND_IMPORTED, None)
 
     def visit_import_all(self, node: ImportAll) -> None:
-        node.is_top_level = True
+        node.is_top_level = self.sem.is_module_scope()
 
     def visit_while_stmt(self, s: WhileStmt) -> None:
-        s.body.accept(self)
-        if s.else_body:
-            s.else_body.accept(self)
+        if self.sem.is_module_scope():
+            s.body.accept(self)
+            if s.else_body:
+                s.else_body.accept(self)
 
     def visit_for_stmt(self, s: ForStmt) -> None:
-        self.analyze_lvalue(s.index)
-        s.body.accept(self)
-        if s.else_body:
-            s.else_body.accept(self)
+        if self.sem.is_module_scope():
+            self.analyze_lvalue(s.index)
+            s.body.accept(self)
+            if s.else_body:
+                s.else_body.accept(self)
 
     def visit_with_stmt(self, s: WithStmt) -> None:
-        for n in s.target:
-            if n:
-                self.analyze_lvalue(n)
-        s.body.accept(self)
+        if self.sem.is_module_scope():
+            for n in s.target:
+                if n:
+                    self.analyze_lvalue(n)
+            s.body.accept(self)
 
     def visit_decorator(self, d: Decorator) -> None:
         d.var._fullname = self.sem.qualified_name(d.var.name())
-        self.sem.add_symbol(d.var.name(), SymbolTableNode(GDEF, d.var), d)
+        self.sem.add_symbol(d.var.name(), SymbolTableNode(self.kind_by_scope(), d.var), d)
 
     def visit_if_stmt(self, s: IfStmt) -> None:
         infer_reachability_of_if_statement(s, pyversion=self.pyversion, platform=self.platform)
@@ -2845,10 +2877,22 @@ def visit_if_stmt(self, s: IfStmt) -> None:
             s.else_body.accept(self)
 
     def visit_try_stmt(self, s: TryStmt) -> None:
-        self.sem.analyze_try_stmt(s, self, add_global=True)
+        if self.sem.is_module_scope():
+            self.sem.analyze_try_stmt(s, self, add_global=self.sem.is_module_scope())
 
     def analyze_lvalue(self, lvalue: Lvalue, explicit_type: bool = False) -> None:
-        self.sem.analyze_lvalue(lvalue, add_global=True, explicit_type=explicit_type)
+        self.sem.analyze_lvalue(lvalue, add_global=self.sem.is_module_scope(),
+                                explicit_type=explicit_type)
+
+    def kind_by_scope(self) -> int:
+        if self.sem.is_module_scope():
+            return GDEF
+        elif self.sem.is_class_scope():
+            return MDEF
+        elif self.sem.is_func_scope():
+            return LDEF
+        else:
+            assert False, "Couldn't determine scope"
 
 
 class ThirdPass(TraverserVisitor):


Test patch:
diff --git a/test-data/unit/check-unreachable-code.test b/test-data/unit/check-unreachable-code.test
--- a/test-data/unit/check-unreachable-code.test
+++ b/test-data/unit/check-unreachable-code.test
@@ -372,7 +372,7 @@ def foo() -> None:
 [out]
 main: note: In function "foo":
 
-[case testSysVersionInfoInMethod]
+[case testSysPlatformInMethod]
 import sys
 class C:
     def foo(self) -> None:
@@ -385,6 +385,19 @@ class C:
 [out]
 main: note: In member "foo" of class "C":
 
+[case testSysPlatformInFunctionImport]
+import sys
+def foo() -> None:
+    if sys.platform != 'fictional':
+        import a
+    else:
+        import b as a
+    a.x
+[file a.py]
+x = 1
+[builtins fixtures/ops.pyi]
+[out]
+
 [case testCustomSysVersionInfo]
 # flags: --python-version 3.2
 import sys