Repository: twisted/twisted

Issue:
Get support for Python 3.11
|[<img alt="adiroiban's avatar" src="https://avatars.githubusercontent.com/u/204609?s=50" width="50" height="50">](https://github.com/adiroiban)| @adiroiban reported|
|-|-|
|Trac ID|trac#10343|
|Type|enhancement|
|Created|2022-05-11 20:27:34Z|

Some info at https://github.com/twisted/twisted/pull/1723#issuecomment-1107654772 from Colin

BTW, I haven't attempted to add test runs for 3.11 yet because there are some other problems.  We need a Cython release with cython/cython[#4667](https://github.com/twisted/twisted/issues/4667), an Automat release with glyph/automat[#7453](https://github.com/twisted/twisted/issues/7453), there are some weird `builtins.TypeError: __dict__ must be set to a dictionary, not a '_Dereference'` test failures that I don't understand, and there are a few fairly trivial test failures due to https://docs.python.org/3.11/whatsnew/3.11.html#enhanced-error-locations-in-tracebacks.  But it's generally looking closer, and this on its own is enough to let me get the testtools test suite passing.

<details><summary>Searchable metadata</summary>

```
trac-id__10343 10343
type__enhancement enhancement
reporter__adiroiban adiroiban
priority__normal normal
milestone__None None
branch__ 
branch_author__ 
status__new new
resolution__None None
component__core core
keywords__None None
time__1652300854237229 1652300854237229
changetime__1652425385864237 1652425385864237
version__None None
owner__None None
cc__mgorny
```
</details>



Discussion:
|[<img alt="mgorny's avatar" src="https://avatars.githubusercontent.com/u/110765?s=50" width="50" height="50">](https://github.com/mgorny)<a name="note_1"></a>|@mgorny commented|
|-|-|

You can run tests using cython from 0.29.x git branch.  In virtualenv it's a bit of PITA because of the new fancy PEP517 logic that insists on building using pypi's current cython but it's doable if you use `setup.py install` instead.
From gitter https://gitter.im/twisted/twisted?at=63529430bad3c73752e7654a

from @DMRobertson

> Hi all. I've been trying to run Synapse in CI against Python 3.11 RCs, but I'm running into some new errors. One class of [errors involves `code` objects](https://github.com/matrix-org/synapse/actions/runs/3275412420/jobs/5390339185#step:7:3808), and I [think](https://github.com/matrix-org/synapse/pull/13812#issuecomment-1247083641) is fixed in recent Twisted releases provided you use a sufficiently new version of `automat`. (See https://github.com/glyph/automat/issues/139#issuecomment-1273146975)
> 
> I reached out [here](https://github.com/glyph/automat/issues/139) to ask if there were plans to make a new release but haven't heard back. Does anyone here know more?



Resolving patch:
diff --git a/src/twisted/persisted/aot.py b/src/twisted/persisted/aot.py
--- a/src/twisted/persisted/aot.py
+++ b/src/twisted/persisted/aot.py
@@ -399,8 +399,10 @@ def unjellyAO(self, ao):
                 inst = klass.__new__(klass)
                 if hasattr(klass, "__setstate__"):
                     self.callAfter(inst.__setstate__, state)
-                else:
+                elif isinstance(state, dict):
                     inst.__dict__ = state
+                else:
+                    inst.__dict__ = state.__getstate__()
                 return inst
 
             elif c is Ref:
diff --git a/src/twisted/spread/flavors.py b/src/twisted/spread/flavors.py
--- a/src/twisted/spread/flavors.py
+++ b/src/twisted/spread/flavors.py
@@ -398,6 +398,8 @@ def setCopyableState(self, state):
         object's dictionary (or a filtered approximation of it depending
         on my peer's perspective).
         """
+        if not state:
+            state = {}
         state = {
             x.decode("utf8") if isinstance(x, bytes) else x: y for x, y in state.items()
         }
diff --git a/src/twisted/spread/jelly.py b/src/twisted/spread/jelly.py
--- a/src/twisted/spread/jelly.py
+++ b/src/twisted/spread/jelly.py
@@ -154,7 +154,8 @@ def _newInstance(cls, state):
     instance = _createBlank(cls)
 
     def defaultSetter(state):
-        instance.__dict__ = state
+        if isinstance(state, dict):
+            instance.__dict__ = state or {}
 
     setter = getattr(instance, "__setstate__", defaultSetter)
     setter(state)


Test patch:
diff --git a/.github/workflows/test.yaml b/.github/workflows/test.yaml
--- a/.github/workflows/test.yaml
+++ b/.github/workflows/test.yaml
@@ -122,8 +122,11 @@ jobs:
           # Just Python 3.9 with default settings.
           - python-version: '3.9'
 
+          # Just Python 3.11 with default settings.
+          - python-version: '3.11'
+
           # Newest macOS and newest Python supported versions.
-          - python-version: '3.10'
+          - python-version: '3.11'
             runs-on: 'macos-12'
             job-name: 'macos-12-default-tests'
 
@@ -138,7 +141,7 @@ jobs:
             trial-args: '--reactor=select'
 
           # Windows, newest Python supported version with iocp reactor.
-          - python-version: '3.10'
+          - python-version: '3.11'
             runs-on: 'windows-2022'
             tox-env: 'alldeps-withcov-windows'
             job-name: 'win-default-tests-iocp'
diff --git a/src/twisted/test/test_persisted.py b/src/twisted/test/test_persisted.py
--- a/src/twisted/test/test_persisted.py
+++ b/src/twisted/test/test_persisted.py
@@ -378,6 +378,10 @@ class UnknownType:
             def __dict__(self):
                 raise AttributeError()
 
+            @property
+            def __getstate__(self):
+                raise AttributeError()
+
         self.assertRaises(TypeError, aot.jellyToSource, UnknownType())
 
     def test_basicIdentity(self):
diff --git a/src/twisted/trial/test/test_pyunitcompat.py b/src/twisted/trial/test/test_pyunitcompat.py
--- a/src/twisted/trial/test/test_pyunitcompat.py
+++ b/src/twisted/trial/test/test_pyunitcompat.py
@@ -218,8 +218,10 @@ def test_tracebackFromCleanFailure(self):
         pyresult = pyunit.TestResult()
         result = PyUnitResultAdapter(pyresult)
         result.addError(self, f)
+        tback = "".join(traceback.format_exception(*exc_info))
         self.assertEqual(
-            pyresult.errors[0][1], "".join(traceback.format_exception(*exc_info))
+            pyresult.errors[0][1].endswith("ZeroDivisionError: division by zero\n"),
+            tback.endswith("ZeroDivisionError: division by zero\n"),
         )
 
     def test_trialSkip(self):
diff --git a/src/twisted/web/test/test_flatten.py b/src/twisted/web/test/test_flatten.py
--- a/src/twisted/web/test/test_flatten.py
+++ b/src/twisted/web/test/test_flatten.py
@@ -706,15 +706,15 @@ def render(self, request: Optional[IRequest]) -> Flattenable:
                     Exception while flattening:
                       \\[<unrenderable>\\]
                       <unrenderable>
-                      .*
+                      <Deferred at .* current result: <twisted.python.failure.Failure builtins.RuntimeError: example>>
                       File ".*", line \\d*, in _flattenTree
-                        element = await element
-                    RuntimeError: example
+                        element = await element.*
                     """
                 ),
                 flags=re.MULTILINE,
             ),
         )
+        self.assertIn("RuntimeError: example", str(failure.value))
         # The original exception is unmodified and will be logged separately if
         # unhandled.
         self.failureResultOf(failing, RuntimeError)