Repository: python/mypy

Issue:
stubtest crash when a referenced module is a DLL
<!--
  Use this form only if mypy reports an "INTERNAL ERROR" and/or gives a traceback.
  Please include the traceback and all other messages below (use `mypy --show-traceback`).
-->

**Crash Report**

stubtest crashes when a referenced module is a DLL, as it seemingly tries to parse the `.dll`'s content as text.

**Traceback**

```
PS C:\Users\Avasam\Documents\Git\typeshed> python .\tests\stubtest_third_party.py pywin32
pywin32... fail

C:\Users\Avasam\AppData\Local\Temp\tmp3yxfa81_\Scripts\pip.exe install pywin32[]==304.* mypy==0.982
MYPYPATH=C:\Users\Avasam\Documents\Git\typeshed\stubs\pywin32 C:\Users\Avasam\AppData\Local\Temp\tmp3yxfa81_\Scripts\python.exe -m mypy.stubtest --custom-typeshed-dir C:\Users\Avasam\Documents\Git\typeshed pythonwin win32 win32comext pythoncom _win32typing --allowlist C:\Users\Avasam\Documents\Git\typeshed\stubs\pywin32\@tests\stubtest_allowlist.txt

Traceback (most recent call last):
  File "C:\Program Files\Python39\lib\tokenize.py", line 330, in find_cookie
    line_string = line.decode('utf-8')
UnicodeDecodeError: 'utf-8' codec can't decode byte 0x90 in position 2: invalid start byte

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "C:\Program Files\Python39\lib\runpy.py", line 197, in _run_module_as_main
    return _run_code(code, main_globals, None,
  File "C:\Program Files\Python39\lib\runpy.py", line 87, in _run_code
    exec(code, run_globals)
  File "C:\Users\Avasam\AppData\Local\Temp\tmp3yxfa81_\lib\site-packages\mypy\stubtest.py", line 1713, in <module>
    sys.exit(main())
  File "C:\Users\Avasam\AppData\Local\Temp\tmp3yxfa81_\lib\site-packages\mypy\stubtest.py", line 1709, in main
    return test_stubs(parse_options(sys.argv[1:]))
  File "C:\Users\Avasam\AppData\Local\Temp\tmp3yxfa81_\lib\site-packages\mypy\stubtest.py", line 1605, in test_stubs
    print(error.get_description(concise=args.concise))
  File "C:\Users\Avasam\AppData\Local\Temp\tmp3yxfa81_\lib\site-packages\mypy\stubtest.py", line 138, in get_description
    runtime_line = inspect.getsourcelines(self.runtime_object)[1]
  File "C:\Program Files\Python39\lib\inspect.py", line 1006, in getsourcelines
    lines, lnum = findsource(object)
  File "C:\Program Files\Python39\lib\inspect.py", line 831, in findsource
    lines = linecache.getlines(file, module.__dict__)
  File "C:\Program Files\Python39\lib\linecache.py", line 46, in getlines
    return updatecache(filename, module_globals)
  File "C:\Program Files\Python39\lib\linecache.py", line 136, in updatecache
    with tokenize.open(fullname) as fp:
  File "C:\Program Files\Python39\lib\tokenize.py", line 394, in open
    encoding, lines = detect_encoding(buffer.readline)
  File "C:\Program Files\Python39\lib\tokenize.py", line 371, in detect_encoding
    encoding = find_cookie(first)
  File "C:\Program Files\Python39\lib\tokenize.py", line 335, in find_cookie
    raise SyntaxError(msg)
SyntaxError: invalid or missing encoding declaration for 'C:\\Users\\Avasam\\AppData\\Local\\Temp\\tmp3yxfa81_\\lib\\site-packages\\pywin32_system32\\pywintypes39.dll'
```

**To Reproduce**

The easiest way to reproduce would be to use the pywin32 stubs from typeshed as an example. It has references to `pywintypes39.dll` and `pythoncom39.dll`. Make sure that `METADATA.toml` has:
```toml
[tool.stubtest]
ignore_missing_stub = false
skip = false
```
`python .\tests\stubtest_third_party.py pywin32`

**Your Environment**

<!-- Include as many relevant details about the environment you experienced the bug in -->

- Mypy version used: 0.981 (compiled: yes)
- Mypy command-line flags: (see above)
- Mypy configuration options from `mypy.ini` (and other config files): (see above)
- Python version used: 3.9.13
- Operating system and version: Windows 10.0.19044 Build 19044


<!--
You can freely edit this text, please remove all the lines
you believe are unnecessary.
-->



Discussion:
Related, very similar error in stubgen as well:
`stubgen -p pythoncom`

Results in:
`Critical error during semantic analysis: mypy: can't decode file '[...]\.venv\Lib\site-packages\pywin32_system32\pythoncom39.dll': 'utf-8' codec can't decode 
byte 0x90 in position 2: invalid start byte`
This feels like a CPython bug, since stubtest is just calling `inspect.getsourcelines`. I think the relevant lines are here: https://github.com/python/cpython/blob/b44372e03c5461b6ad3d89763a9eb6cb82df07a4/Lib/inspect.py#L946

Basically, inspect doesn't try to find a source file if the object's `__file__` has one of the `importlib.machinery.EXTENSION_SUFFIXES`. But in this case, I suppose pywin32 exposes an object with a `__file__` that points to a .dll.

Where should this be fixed?
- Maybe CPython should try harder to avoid loading binary files as source files. But I don't know if that's going to be fully reliable, and people do load Python code from files with non-`.py` extension files.
- Maybe pywin32 shouldn't set a `__file__` pointing to a file that's not a valid extension module.
- Maybe stubtest should be more resilient to `getsourcelines()` failing. That's not very principled, but it's good for stubtest to be resilient to odd failures, so I think we should do that.

Resolving patch:
diff --git a/mypy/stubtest.py b/mypy/stubtest.py
--- a/mypy/stubtest.py
+++ b/mypy/stubtest.py
@@ -136,7 +136,7 @@ def get_description(self, concise: bool = False) -> str:
         if not isinstance(self.runtime_object, Missing):
             try:
                 runtime_line = inspect.getsourcelines(self.runtime_object)[1]
-            except (OSError, TypeError):
+            except (OSError, TypeError, SyntaxError):
                 pass
             try:
                 runtime_file = inspect.getsourcefile(self.runtime_object)


Test patch: