Repository: python/mypy

Issue:
mypy incremental issues with `--warn-unused-ignores`
I just had the following sequence happen, which suggests to me that the mypy cache is broken with this option:

```
(zulip-py3-venv) tabbott@zaset:~/zulip$ ./tools/run-mypy 
(zulip-py3-venv) tabbott@zaset:~/zulip$ ./tools/run-mypy --py3
zerver/lib/cache.py:287: note: unused 'type: ignore' comment
zproject/jinja2/__init__.py:32: note: unused 'type: ignore' comment
zerver/tornado/event_queue.py:474: note: unused 'type: ignore' comment
zerver/lib/actions.py:1199: note: unused 'type: ignore' comment
(zulip-py3-venv) tabbott@zaset:~/zulip$ ./tools/run-mypy --py3^C
(zulip-py3-venv) tabbott@zaset:~/zulip$ rm -rf var/mypy-cache/
(zulip-py3-venv) tabbott@zaset:~/zulip$ ./tools/run-mypy 
api/integrations/asana/zulip_asana_mirror:81: note: unused 'type: ignore' comment
api/integrations/asana/zulip_asana_mirror:82: note: unused 'type: ignore' comment
api/integrations/google/google-calendar:133: note: unused 'type: ignore' comment
api/integrations/google/google-calendar:143: note: unused 'type: ignore' comment
scripts/setup/generate_secrets.py:54: note: unused 'type: ignore' comment
tools/replace-tarball-shebang:22: note: unused 'type: ignore' comment
tools/show-profile-results.py:22: note: unused 'type: ignore' comment
tools/tests/test_template_parser.py:31: note: unused 'type: ignore' comment
zerver/lib/actions.py:1199: note: unused 'type: ignore' comment
zerver/lib/camo.py:18: note: unused 'type: ignore' comment
zerver/lib/parallel.py:32: note: unused 'type: ignore' comment
zerver/lib/test_helpers.py:175: note: unused 'type: ignore' comment
zerver/management/commands/email_mirror.py:112: note: unused 'type: ignore' comment
zerver/management/commands/email_mirror.py:119: note: unused 'type: ignore' comment
zerver/management/commands/email_mirror.py:128: note: unused 'type: ignore' comment
zerver/storage.py:45: note: unused 'type: ignore' comment
zerver/storage.py:75: note: unused 'type: ignore' comment
zerver/tests/test_upload.py:746: note: unused 'type: ignore' comment
zerver/tornado/ioloop_logging.py:24: note: unused 'type: ignore' comment
zproject/jinja2/__init__.py:32: note: unused 'type: ignore' comment
zproject/local_settings.py:14: note: unused 'type: ignore' comment
(zulip-py3-venv) tabbott@zaset:~/zulip$ ./tools/run-mypy 
(zulip-py3-venv) tabbott@zaset:~/zulip$
```


Discussion:
I think what happens here is that the data for `--warn-unused-ignores` is only collected when it parses a file, not when it reads the data for a file from the cache. So I guess in your first run some files were already in the cache.

I don't believe there's a bug to be fixed here (or at least I would prefer not to have to serialize the `# type: ignore` map into the cache). Maybe we can add a note to the docs.
Hmm.  Our model for how we use mypy is that we want to run it as part of our linter suite, along with pyflakes and friends (with the caching, it's basically fast enough).  And the easy thing to do there is to just pass the `-i` option to `mypy` unconditionally (we do that today).  

But if we do that, then suddenly this flag doesn't work at all, unless you know to delete the mypy cache each time you want to run it.

So I guess my vote is for updating the cache to include that map.
I hear you -- but supporting this use case is a low priority for us.
I've been able to work around this for CI by making sure mypy_cache is deleted between runs. 
I am deep in a rabbit hole of trying to understand if there are going to be any unexpected ramifications to a change to the handling of `suppressed` that I want to make, and in the process I produced these two test cases that I believe are for this bug:

```
[case testIgnoreWarning1]
# cmd: mypy -m a b
# cmd2: mypy -m a b
# cmd3: mypy -m a b c
# flags: --follow-imports=error --warn-unused-ignores --verbose
[file a.py]
import b
import c  # type: ignore
[file b.py]
x = 1
[file b.py.2]
x = 'hi'
[file c.py]
pass
[out]
[out2]
[out3]
tmp/a.py:1: note: unused 'type: ignore' comment

[case testIgnoreWarning2]
# flags: --warn-unused-ignores --verbose
import a
[file a.py]
import b
import c  # type: ignore
[file b.py]
x = 1
[file b.py.2]
x = 'hi'
[file c.py.3]
pass
[out]
[out2]
[out3]
tmp/a.py:1: note: unused 'type: ignore' comment
```

They both fail in stage 2 with spurious unused type:ignore warnings.
We may want to address this somehow (perhaps just by disallowing the combination for now) before making incremental the default mode per #4800.

The snag here also does not actually seem to be that we don't write out the ignore errors map (which I think is fine), but that we don't generate error messages for failed imports from modules that are loaded from the cache initially but then later parsed due to changes in their dependencies.

Resolving patch:
diff --git a/mypy/build.py b/mypy/build.py
--- a/mypy/build.py
+++ b/mypy/build.py
@@ -396,6 +396,8 @@ def default_lib_path(data_dir: str,
                         ('suppressed', List[str]),  # dependencies that weren't imported
                         ('child_modules', List[str]),  # all submodules of the given module
                         ('options', Optional[Dict[str, object]]),  # build options
+                        # dep_prios and dep_lines are in parallel with
+                        # dependencies + suppressed.
                         ('dep_prios', List[int]),
                         ('dep_lines', List[int]),
                         ('interface_hash', str),  # hash representing the public interface
@@ -964,8 +966,8 @@ def find_cache_meta(id: str, path: str, manager: BuildManager) -> Optional[Cache
     # Ignore cache if generated by an older mypy version.
     if ((m.version_id != manager.version_id and not manager.options.skip_version_check)
             or m.options is None
-            or len(m.dependencies) != len(m.dep_prios)
-            or len(m.dependencies) != len(m.dep_lines)):
+            or len(m.dependencies) + len(m.suppressed) != len(m.dep_prios)
+            or len(m.dependencies) + len(m.suppressed) != len(m.dep_lines)):
         manager.log('Metadata abandoned for {}: new attributes are missing'.format(id))
         return None
 
@@ -1514,12 +1516,13 @@ def __init__(self,
             # compare them to the originals later.
             self.dependencies = list(self.meta.dependencies)
             self.suppressed = list(self.meta.suppressed)
-            assert len(self.meta.dependencies) == len(self.meta.dep_prios)
+            all_deps = self.dependencies + self.suppressed
+            assert len(all_deps) == len(self.meta.dep_prios)
             self.priorities = {id: pri
-                               for id, pri in zip(self.meta.dependencies, self.meta.dep_prios)}
-            assert len(self.meta.dependencies) == len(self.meta.dep_lines)
+                               for id, pri in zip(all_deps, self.meta.dep_prios)}
+            assert len(all_deps) == len(self.meta.dep_lines)
             self.dep_line_map = {id: line
-                                 for id, line in zip(self.meta.dependencies, self.meta.dep_lines)}
+                                 for id, line in zip(all_deps, self.meta.dep_lines)}
             self.child_modules = set(self.meta.child_modules)
         else:
             # When doing a fine-grained cache load, pretend we only
@@ -1909,14 +1912,21 @@ def write_cache(self) -> None:
             self.mark_interface_stale()
             self.interface_hash = new_interface_hash
 
-    def verify_dependencies(self) -> None:
-        """Report errors for import targets in modules that don't exist."""
-        # Strip out indirect dependencies. See comment in build.load_graph().
+    def verify_dependencies(self, suppressed_only: bool = False) -> None:
+        """Report errors for import targets in modules that don't exist.
+
+        If suppressed_only is set, only check suppressed dependencies.
+        """
         manager = self.manager
-        dependencies = [dep for dep in self.dependencies
-                        if self.priorities.get(dep) != PRI_INDIRECT]
         assert self.ancestors is not None
-        for dep in dependencies + self.suppressed + self.ancestors:
+        if suppressed_only:
+            all_deps = self.suppressed
+        else:
+            # Strip out indirect dependencies. See comment in build.load_graph().
+            dependencies = [dep for dep in self.dependencies
+                            if self.priorities.get(dep) != PRI_INDIRECT]
+            all_deps = dependencies + self.suppressed + self.ancestors
+        for dep in all_deps:
             options = manager.options.clone_for_module(dep)
             if dep not in manager.modules and not options.ignore_missing_imports:
                 line = self.dep_line_map.get(dep, 1)
@@ -1939,13 +1949,18 @@ def verify_dependencies(self) -> None:
                     pass
 
     def dependency_priorities(self) -> List[int]:
-        return [self.priorities.get(dep, PRI_HIGH) for dep in self.dependencies]
+        return [self.priorities.get(dep, PRI_HIGH) for dep in self.dependencies + self.suppressed]
 
     def dependency_lines(self) -> List[int]:
-        return [self.dep_line_map.get(dep, 1) for dep in self.dependencies]
+        return [self.dep_line_map.get(dep, 1) for dep in self.dependencies + self.suppressed]
 
     def generate_unused_ignore_notes(self) -> None:
         if self.options.warn_unused_ignores:
+            # If this file was initially loaded from the cache, it may have suppressed
+            # dependencies due to imports with ignores on them. We need to generate
+            # those errors to avoid spuriously flagging them as unused ignores.
+            if self.meta:
+                self.verify_dependencies(suppressed_only=True)
             self.manager.errors.generate_unused_ignore_notes(self.xpath)
 
 


Test patch:
diff --git a/test-data/unit/check-incremental.test b/test-data/unit/check-incremental.test
--- a/test-data/unit/check-incremental.test
+++ b/test-data/unit/check-incremental.test
@@ -28,7 +28,7 @@
 -- annotation, and any files expect to be stale (aka have a modified interface)
 -- should be annotated in the [stale] annotation. Note that a file that ends up
 -- producing an error has its caches deleted and is marked stale automatically.
--- Such files don't need to be included in [stale ...] list.
+-- Such files do not need to be included in [stale ...] list.
 --
 -- The test suite will automatically assume that __main__ is stale and rechecked in
 -- all cases so we can avoid constantly having to annotate it. The list of
@@ -4206,3 +4206,32 @@ class Two:
     pass
 [out2]
 tmp/m/two.py:2: error: Revealed type is 'builtins.str'
+
+[case testImportUnusedIgnore1]
+# flags: --warn-unused-ignores
+import a
+[file a.py]
+import b
+import foo  # type: ignore
+[file b.py]
+x = 1
+[file b.py.2]
+x = '2'
+
+-- TODO: Fails due to #4856
+[case testImportUnusedIgnore2-skip]
+# flags: --warn-unused-ignores
+import a
+[file a.py]
+import b
+import c  # type: ignore
+[file b.py]
+x = 1
+[file b.py.2]
+x = 'hi'
+[file c.py.3]
+pass
+[out]
+[out2]
+[out3]
+tmp/a.py:1: note: unused 'type: ignore' comment