Repository: aio-libs/aiosmtpd (python)

Issue:
Do not call `asyncio.get_event_loop()` if no existing event loop
As described in https://docs.python.org/3/library/asyncio-eventloop.html , directly calling `asyncio.get_event_loop()` raises DeprecationWarning if no event loop is already running. The behavior of `get_event_loop()` was changed in Python 3.10.

Up to Python 3.9, `get_event_loop()` will silently create an event loop with no warnings.

_It seems that_ in Python 3.10 and 3.11, `get_event_loop()` will still create an event loop, but a DeprecationWarning is raised.

The `get_event_loop()` behavior will change completely in Python 3.12, no longer creating an event loop but will raise an Error instead.

We must parse the code and ensure that `get_event_loop()` expecting an implicit event loop creation are removed from the code.


Pull request:
Workaround for asyncio.get_event_loop() deprecation
<!-- Thank you for your contribution! -->

## What do these changes do?

Implement workaround due to the deprecation in ***behavior*** of asyncio.get_event_loop() (which will outright result in error with Python 3.12)

This change of behavior first appeared in Python 3.10: https://docs.python.org/3.10/library/asyncio-eventloop.html#asyncio.get_event_loop

And there has been some discussion on this, among others:
* https://github.com/python/cpython/issues/83710
* https://github.com/python/cpython/issues/93453
* https://github.com/python/cpython/pull/98440

## Are there changes in behavior for the user?

Should be minimal.
* If user creates their own event loop and feeds that to aiosmtpd, then nothing will be changed
* If user does not create their own event loop and let aiosmtpd generates one for them, then the behavior totally depends on how `get_event_loop_policy()` was in effect at time of instantiation.
  *  This might or might not be a problem. For asyncio, the behaviour should be similar. I am not sure about the behavior of other implementations (e.g., `uvloop`)

## Related issue number

Closes #353 

## Checklist

- [x] I think the code is well written
- [x] Unit tests for the changes exist
- [x] tox testenvs have been executed in the following environments:
  - [x] Windows (10): `py311-{nocov,cov}`
- [x] Documentation reflects the changes


Patch:
diff --git a/README.rst b/README.rst
index 6c2e018..2714318 100644
--- a/README.rst
+++ b/README.rst
@@ -155,11 +155,11 @@ option::
 
 You can also add the ``-s``/``--capture=no`` option to show output, e.g.::
 
-    $ tox -e py36-nocov -- -s
+    $ tox -e py37-nocov -- -s
 
 and these options can be combined::
 
-    $ tox -e py36-nocov -- -x -s <testname>
+    $ tox -e py37-nocov -- -x -s <testname>
 
 (The ``-e`` parameter is explained in the next section about 'testenvs'.
 In general, you'll want to choose the ``nocov`` testenvs if you want to show output,
@@ -173,7 +173,7 @@ In general, the ``-e`` parameter to tox specifies one (or more) **testenv**
 to run (separate using comma if more than one testenv). The following testenvs
 have been configured and tested:
 
-* ``{py37,py38,py39,py310,py311,pypy3}-{nocov,cov,diffcov,profile}``
+* ``{py37,py38,py39,py310,py311,pypy3,pypy37,pypy38,pypy39}-{nocov,cov,diffcov,profile}``
 
   Specifies the interpreter to run and the kind of testing to perform.
 
diff --git a/aiosmtpd/__init__.py b/aiosmtpd/__init__.py
index 45c4341..7431874 100644
--- a/aiosmtpd/__init__.py
+++ b/aiosmtpd/__init__.py
@@ -1,4 +1,21 @@
 # Copyright 2014-2021 The aiosmtpd Developers
 # SPDX-License-Identifier: Apache-2.0
+import asyncio
+import warnings
 
-__version__ = "1.4.4a0"
+
+__version__ = "1.4.4a1"
+
+
+def _get_or_new_eventloop() -> asyncio.AbstractEventLoop:
+    loop = None
+    with warnings.catch_warnings():
+        warnings.simplefilter("error")
+        try:
+            loop = asyncio.get_event_loop()
+        except (DeprecationWarning, RuntimeError):  # pragma: py-lt-310
+            if loop is None:  # pragma: py-lt-312
+                loop = asyncio.new_event_loop()
+                asyncio.set_event_loop(loop)
+    assert isinstance(loop, asyncio.AbstractEventLoop)
+    return loop
diff --git a/aiosmtpd/docs/NEWS.rst b/aiosmtpd/docs/NEWS.rst
index 3154297..140151f 100644
--- a/aiosmtpd/docs/NEWS.rst
+++ b/aiosmtpd/docs/NEWS.rst
@@ -16,10 +16,12 @@ Fixed/Improved
 * A whole bunch of annotations
 
 
-1.4.4a0 (ad hoc)
+1.4.4a1 (ad hoc)
 ================
 
-(Stub ``NEWS.rst`` entry as placeholder for ``qa`` test.)
+Fixed/Improved
+--------------
+* No longer expect an implicit creation of the event loop through ``get_event_loop()`` (Closes #353)
 
 
 1.4.3 (2022-12-21)
diff --git a/aiosmtpd/docs/controller.rst b/aiosmtpd/docs/controller.rst
index 2fb81c0..220d45a 100644
--- a/aiosmtpd/docs/controller.rst
+++ b/aiosmtpd/docs/controller.rst
@@ -236,7 +236,8 @@ you'll have to do something similar to this:
 .. doctest:: unthreaded
 
     >>> import asyncio
-    >>> loop = asyncio.get_event_loop()
+    >>> loop = asyncio.new_event_loop()
+    >>> asyncio.set_event_loop(loop)
     >>> from aiosmtpd.controller import UnthreadedController
     >>> from aiosmtpd.handlers import Sink
     >>> controller = UnthreadedController(Sink(), loop=loop)
diff --git a/aiosmtpd/handlers.py b/aiosmtpd/handlers.py
index 5e44f7c..2d1b28f 100644
--- a/aiosmtpd/handlers.py
+++ b/aiosmtpd/handlers.py
@@ -25,6 +25,7 @@ from typing import Any, AnyStr, List, Type, TypeVar, Optional
 
 from public import public
 
+from aiosmtpd import _get_or_new_eventloop
 from aiosmtpd.smtp import SMTP as SMTPServer
 from aiosmtpd.smtp import Envelope as SMTPEnvelope
 from aiosmtpd.smtp import Session as SMTPSession
@@ -218,7 +219,7 @@ class AsyncMessage(Message, metaclass=ABCMeta):
         loop: Optional[asyncio.AbstractEventLoop] = None,
     ):
         super().__init__(message_class)
-        self.loop = loop or asyncio.get_event_loop()
+        self.loop = loop or _get_or_new_eventloop()
 
     async def handle_DATA(
         self, server: SMTPServer, session: SMTPSession, envelope: SMTPEnvelope
diff --git a/aiosmtpd/main.py b/aiosmtpd/main.py
index 2366ae4..166484c 100644
--- a/aiosmtpd/main.py
+++ b/aiosmtpd/main.py
@@ -1,7 +1,6 @@
 # Copyright 2014-2021 The aiosmtpd Developers
 # SPDX-License-Identifier: Apache-2.0
 
-import asyncio
 import logging
 import os
 import signal
@@ -16,7 +15,7 @@ from typing import Optional, Sequence, Tuple
 
 from public import public
 
-from aiosmtpd import __version__
+from aiosmtpd import __version__, _get_or_new_eventloop
 from aiosmtpd.smtp import DATA_SIZE_DEFAULT, SMTP
 
 try:
@@ -252,7 +251,7 @@ def main(args: Optional[Sequence[str]] = None) -> None:
 
     logging.basicConfig(level=logging.ERROR)
     log = logging.getLogger("mail.log")
-    loop = asyncio.get_event_loop()
+    loop = _get_or_new_eventloop()
 
     if args.debug > 0:
         log.setLevel(logging.INFO)
diff --git a/aiosmtpd/smtp.py b/aiosmtpd/smtp.py
index 95181ea..a977f75 100644
--- a/aiosmtpd/smtp.py
+++ b/aiosmtpd/smtp.py
@@ -34,7 +34,7 @@ from warnings import warn
 import attr
 from public import public
 
-from aiosmtpd import __version__
+from aiosmtpd import __version__, _get_or_new_eventloop
 from aiosmtpd.proxy_protocol import ProxyData, get_proxy
 
 
@@ -208,7 +208,7 @@ class Envelope:
 # unit test suite.  In that case, this function is mocked to set the debug
 # level on the loop (as if PYTHONASYNCIODEBUG=1 were set).
 def make_loop() -> asyncio.AbstractEventLoop:
-    return asyncio.get_event_loop()
+    return _get_or_new_eventloop()
 
 
 @public
diff --git a/examples/authenticated_relayer/server.py b/examples/authenticated_relayer/server.py
index c44160c..f00b602 100644
--- a/examples/authenticated_relayer/server.py
+++ b/examples/authenticated_relayer/server.py
@@ -94,7 +94,8 @@ if __name__ == '__main__':
         print(f"Please create {DB_AUTH} first using make_user_db.py")
         sys.exit(1)
     logging.basicConfig(level=logging.DEBUG)
-    loop = asyncio.get_event_loop()
+    loop = asyncio.new_event_loop()
+    asyncio.set_event_loop(loop)
     loop.create_task(amain())
     try:
         loop.run_forever()
diff --git a/examples/basic/server.py b/examples/basic/server.py
index 9f6f714..fcb572a 100644
--- a/examples/basic/server.py
+++ b/examples/basic/server.py
@@ -15,7 +15,8 @@ async def amain(loop):
 
 if __name__ == '__main__':
     logging.basicConfig(level=logging.DEBUG)
-    loop = asyncio.get_event_loop()
+    loop = asyncio.new_event_loop()
+    asyncio.set_event_loop(loop)
     loop.create_task(amain(loop=loop))
     try:
         loop.run_forever()
diff --git a/pyproject.toml b/pyproject.toml
index e067d36..8731e60 100644
--- a/pyproject.toml
+++ b/pyproject.toml
@@ -35,6 +35,8 @@ source = [
 
 [tool.coverage.coverage_conditional_plugin.rules]
 # Here we specify our pragma rules:
+py-lt-312 = "sys_version_info < (3, 12)"
+py-lt-310 = "sys_version_info < (3, 10)"
 py-ge-38 = "sys_version_info >= (3, 8)"
 py-lt-38 = "sys_version_info < (3, 8)"
 py-gt-36 = "sys_version_info > (3, 6)"
diff --git a/tox.ini b/tox.ini
index f4fcf92..211e78d 100644
--- a/tox.ini
+++ b/tox.ini
@@ -4,15 +4,7 @@ envlist = qa, static, docs, py{37,38,39,310,311,py3}-{nocov,cov,diffcov}
 skip_missing_interpreters = True
 
 [testenv]
-# One virtualenv per Python version
-envdir =
-    py37: {toxworkdir}/3.7
-    py38: {toxworkdir}/3.8
-    py39: {toxworkdir}/3.9
-    py310: {toxworkdir}/3.10
-    py311: {toxworkdir}/3.11
-    pypy3: {toxworkdir}/pypy3
-    py: {toxworkdir}/py
+envdir = {toxworkdir}/{envname}
 commands =
     python housekeep.py prep
     # Bandit is not needed on diffcov, and seems to be incompatible with 310
@@ -27,20 +19,20 @@ commands =
 #sitepackages = True
 usedevelop = True
 deps =
-    # do NOT make these conditional, that way we can reuse same envdir for nocov+cov+diffcov
     bandit
     colorama
-    coverage[toml]
-    coverage-conditional-plugin
     packaging
     pytest >= 6.0  # Require >= 6.0 for pyproject.toml support (PEP 517)
-    pytest-cov
     pytest-mock
     pytest-print
     pytest-profiling
     pytest-sugar
     py # needed for pytest-sugar as it doesn't declare dependency on it.
-    diff_cover
+    !nocov: coverage>=7.0.1
+    !nocov: coverage[toml]
+    !nocov: coverage-conditional-plugin
+    !nocov: pytest-cov
+    diffcov: diff_cover
 setenv =
     cov: COVERAGE_FILE={toxinidir}/_dump/.coverage
     nocov: PYTHONASYNCIODEBUG=1
@@ -50,6 +42,8 @@ setenv =
     py310: INTERP=py310
     py311: INTERP=py311
     pypy3: INTERP=pypy3
+    pypy37: INTERP=pypy37
+    pypy38: INTERP=pypy38
     py: INTERP=py
 passenv =
     PYTHON*


Test patch:
diff --git a/.github/workflows/unit-testing-and-coverage.yml b/.github/workflows/unit-testing-and-coverage.yml
index 6e92f7e..e867f07 100644
--- a/.github/workflows/unit-testing-and-coverage.yml
+++ b/.github/workflows/unit-testing-and-coverage.yml
@@ -90,17 +90,8 @@ jobs:
       # If a matrix fail, do NOT stop other matrix, let them run to completion
       fail-fast: false
       matrix:
-        python-version: [ "3.7", "3.8", "3.9", "3.10", "3.11", "pypy3.6", "pypy3.7", "pypy3.8" ]
+        python-version: [ "3.7", "3.8", "3.9", "3.10", "3.11", "pypy3.7", "pypy3.8", "pypy3.9" ]
         os: [ "macos-11", "macos-12", "ubuntu-18.04", "ubuntu-20.04", "ubuntu-22.04", "windows-2019", "windows-2022" ]
-        exclude:
-          - os: windows-2019
-            python-version: pypy3.6
-          - os: windows-2022
-            python-version: pypy3.6
-          - os: macos-11
-            python-version: pypy3.6
-          - os: macos-12
-            python-version: pypy3.6
     runs-on: ${{ matrix.os }}
     timeout-minutes: 15  # Slowest so far is pypy3 on MacOS, taking almost 7m
     steps:
@@ -117,7 +108,7 @@ jobs:
       run: |
         python -m pip install --upgrade pip setuptools wheel
         # Test deps
-        pip install colorama coverage[toml] "coverage-conditional-plugin>=0.5.0" packaging pytest pytest-cov pytest-mock
+        pip install colorama "coverage>=7.0.1" coverage[toml] "coverage-conditional-plugin>=0.5.0" packaging pytest pytest-cov pytest-mock
         # Package deps
         python setup.py develop
     - name: "Security checking"
@@ -128,6 +119,8 @@ jobs:
 
     # IMPORTANT: pypy3.8 is currently excluded from coverage testing because coverage seems to be unstable when
     # running on PyPy 3.8. This is still under investigation (See issue #325)
+    # Edit: This is due to change in behavior in PyPy v7.3.10, and coverage.py needs to adapt.
+    # See: https://github.com/nedbat/coveragepy/issues/1515
 
     - name: "Execute testing with coverage"
       if: matrix.python-version != 'pypy3.8'
diff --git a/aiosmtpd/tests/conftest.py b/aiosmtpd/tests/conftest.py
index 6a8c3dd..0c69103 100644
--- a/aiosmtpd/tests/conftest.py
+++ b/aiosmtpd/tests/conftest.py
@@ -5,6 +5,7 @@ import asyncio
 import inspect
 import socket
 import ssl
+import warnings
 from contextlib import suppress
 from functools import wraps
 from smtplib import SMTP as SMTPClient
@@ -200,14 +201,20 @@ def get_handler(request: pytest.FixtureRequest) -> Callable:
 
 @pytest.fixture
 def temp_event_loop() -> Generator[asyncio.AbstractEventLoop, None, None]:
-    default_loop = asyncio.get_event_loop()
+    with warnings.catch_warnings():
+        warnings.simplefilter("ignore")
+        try:
+            default_loop = asyncio.get_event_loop()
+        except (DeprecationWarning, RuntimeError):
+            default_loop = None
     new_loop = asyncio.new_event_loop()
     asyncio.set_event_loop(new_loop)
     #
     yield new_loop
     #
     new_loop.close()
-    asyncio.set_event_loop(default_loop)
+    if default_loop is not None:
+        asyncio.set_event_loop(default_loop)
 
 
 @pytest.fixture
diff --git a/aiosmtpd/tests/test_misc.py b/aiosmtpd/tests/test_misc.py
new file mode 100644
index 0000000..94f1489
--- /dev/null
+++ b/aiosmtpd/tests/test_misc.py
@@ -0,0 +1,52 @@
+# Copyright 2014-2021 The aiosmtpd Developers
+# SPDX-License-Identifier: Apache-2.0
+
+"""Test other aspects of the server implementation."""
+
+import asyncio
+import warnings
+from typing import Generator, Optional
+
+import pytest
+
+from aiosmtpd import _get_or_new_eventloop
+
+
+@pytest.fixture(scope="module")
+def close_existing_loop() -> Generator[Optional[asyncio.AbstractEventLoop], None, None]:
+    loop: Optional[asyncio.AbstractEventLoop]
+    with warnings.catch_warnings():
+        warnings.filterwarnings("error")
+        try:
+            loop = asyncio.get_event_loop()
+        except (DeprecationWarning, RuntimeError):
+            loop = None
+    if loop:
+        loop.stop()
+        loop.close()
+        asyncio.set_event_loop(None)
+        yield loop
+    else:
+        yield None
+
+
+class TestInit:
+
+    def test_create_new_if_none(self, close_existing_loop):
+        old_loop = close_existing_loop
+        loop: Optional[asyncio.AbstractEventLoop]
+        loop = _get_or_new_eventloop()
+        assert loop is not None
+        assert loop is not old_loop
+        assert isinstance(loop, asyncio.AbstractEventLoop)
+
+    def test_not_create_new_if_exist(self, close_existing_loop):
+        old_loop = close_existing_loop
+        loop: Optional[asyncio.AbstractEventLoop]
+        loop = asyncio.new_event_loop()
+        assert loop is not old_loop
+        asyncio.set_event_loop(loop)
+        ret_loop = _get_or_new_eventloop()
+        assert ret_loop is not old_loop
+        assert ret_loop == loop
+        assert ret_loop is loop