Repository: mitmproxy/mitmproxy

Issue:
TypeError: Subscripted generics cannot be used with class and instance checks under python 3.9.0b1
#### Problem Description
Running mitmproxy 5.1.1 under python 3.9.0b1 fails with `TypeError: Subscripted generics cannot be used with class and instance checks`. The test suite fails as well with hundreds of ERROR and FAILED tests.

#### Steps to reproduce the behavior:
1. install mitmproxy 5.1.1 on Fedora rawhide
2. mitmproxy
3. pytest -v

There are:
```
=================== 303 failed, 994 passed, 2 xfailed, 115 warnings, 182 errors in 72.86s (0:01:12) ====================
```
Most of them throw a `TypeError: Subscripted generics cannot be used with class and instance checks` and have a stack trace similar to:
```
___________________________________ ERROR at setup of TestHTTPS.test_clientcert_dir ____________________________________

cls = <class 'test.mitmproxy.proxy.test_server.TestHTTPS'>

    @classmethod
    def setup_class(cls):
        cls.server = pathod.test.Daemon(
            ssl=cls.ssl,
            ssloptions=cls.ssloptions)
        cls.server2 = pathod.test.Daemon(
            ssl=cls.ssl,
            ssloptions=cls.ssloptions)
    
>       cls.options = cls.get_options()

test/mitmproxy/tservers.py:146: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
test/mitmproxy/tservers.py:179: in get_options
    return options.Options(
mitmproxy/options.py:50: in __init__
    self.add_option(
mitmproxy/optmanager.py:109: in add_option
    self._options[name] = _Option(name, typespec, default, help, choices)
mitmproxy/optmanager.py:34: in __init__
    typecheck.check_option_type(name, default, typespec)
mitmproxy/utils/typecheck.py:73: in check_option_type
    elif not isinstance(value, typeinfo):
/usr/lib64/python3.9/typing.py:649: in __instancecheck__
    return self.__subclasscheck__(type(obj))
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

self = typing.Optional[str], cls = <class 'NoneType'>

    def __subclasscheck__(self, cls):
>       raise TypeError("Subscripted generics cannot be used with"
                        " class and instance checks")
E       TypeError: Subscripted generics cannot be used with class and instance checks

/usr/lib64/python3.9/typing.py:652: TypeError
```

#### System Information
```
Traceback (most recent call last):
  File "/builddir/build/BUILDROOT/mitmproxy-5.1.1-1.fc33.x86_64/usr/bin/./mitmproxy", line 11, in <module>
    load_entry_point('mitmproxy==5.1.1', 'console_scripts', 'mitmproxy')()
  File "/builddir/build/BUILDROOT/mitmproxy-5.1.1-1.fc33.x86_64/usr/lib/python3.9/site-packages/mitmproxy/tools/_main.py", line 147, in mitmproxy
    run(console.master.ConsoleMaster, cmdline.mitmproxy, args)
  File "/builddir/build/BUILDROOT/mitmproxy-5.1.1-1.fc33.x86_64/usr/lib/python3.9/site-packages/mitmproxy/tools/_main.py", line 71, in run
    opts = options.Options()
  File "/builddir/build/BUILDROOT/mitmproxy-5.1.1-1.fc33.x86_64/usr/lib/python3.9/site-packages/mitmproxy/options.py", line 50, in __init__
    self.add_option(
  File "/builddir/build/BUILDROOT/mitmproxy-5.1.1-1.fc33.x86_64/usr/lib/python3.9/site-packages/mitmproxy/optmanager.py", line 109, in add_option
    self._options[name] = _Option(name, typespec, default, help, choices)
  File "/builddir/build/BUILDROOT/mitmproxy-5.1.1-1.fc33.x86_64/usr/lib/python3.9/site-packages/mitmproxy/optmanager.py", line 34, in __init__
    typecheck.check_option_type(name, default, typespec)
  File "/builddir/build/BUILDROOT/mitmproxy-5.1.1-1.fc33.x86_64/usr/lib/python3.9/site-packages/mitmproxy/utils/typecheck.py", line 73, in check_option_type
    elif not isinstance(value, typeinfo):
  File "/usr/lib64/python3.9/typing.py", line 649, in __instancecheck__
    return self.__subclasscheck__(type(obj))
  File "/usr/lib64/python3.9/typing.py", line 652, in __subclasscheck__
    raise TypeError("Subscripted generics cannot be used with"
TypeError: Subscripted generics cannot be used with class and instance checks
```


Discussion:
Ah, wonderful. Thanks for the early heads-up! 😃  Looks like we need to adjust our type-checking for 3.9. I presume we want to wait with this until GitHub Actions allows us to run CI for _some_ Python 3.9 release.
Small Update: #4174 introduces CI coverage for Python 3.9, now it's a matter of fixing the actual issue. :)

Resolving patch:
diff --git a/mitmproxy/utils/typecheck.py b/mitmproxy/utils/typecheck.py
--- a/mitmproxy/utils/typecheck.py
+++ b/mitmproxy/utils/typecheck.py
@@ -39,7 +39,7 @@ def check_option_type(name: str, value: typing.Any, typeinfo: Type) -> None:
 
     typename = str(typeinfo)
 
-    if typename.startswith("typing.Union"):
+    if typename.startswith("typing.Union") or typename.startswith("typing.Optional"):
         for T in union_types(typeinfo):
             try:
                 check_option_type(name, value, T)


Test patch: