Spaces:

ITookAPill
/

PyComp

Running on Zero

Setting the PYTHONASYNCIODEBUG{.interpreted-text role="envvar"} environment variable to 1.
Using the Python Development Mode <devmode>{.interpreted-text role="ref"}.
Passing debug=True to asyncio.run{.interpreted-text role="func"}.
Calling loop.set_debug{.interpreted-text role="meth"}.

In addition to enabling the debug mode, consider also:

setting the log level of the asyncio logger <asyncio-logger>{.interpreted-text role="ref"} to logging.DEBUG{.interpreted-text role="py:const"}, for example the following snippet of code can be run at startup of the application:
```
logging.basicConfig(level=logging.DEBUG)
```
configuring the warnings{.interpreted-text role="mod"} module to display ResourceWarning{.interpreted-text role="exc"} warnings. One way of doing that is by using the -W{.interpreted-text role="option"} default command line option.

When the debug mode is enabled:

Many non-threadsafe asyncio APIs (such as loop.call_soon{.interpreted-text role="meth"} and loop.call_at{.interpreted-text role="meth"} methods) raise an exception if they are called from a wrong thread.
The execution time of the I/O selector is logged if it takes too long to perform an I/O operation.
Callbacks taking longer than 100 milliseconds are logged. The loop.slow_callback_duration{.interpreted-text role="attr"} attribute can be used to set the minimum execution duration in seconds that is considered "slow".

Concurrency and Multithreading {#asyncio-multithreading}

An event loop runs in a thread (typically the main thread) and executes all callbacks and Tasks in its thread. While a Task is running in the event loop, no other Tasks can run in the same thread. When a Task executes an await expression, the running Task gets suspended, and the event loop executes the next Task.

To schedule a callback{.interpreted-text role="term"} from another OS thread, the loop.call_soon_threadsafe{.interpreted-text role="meth"} method should be used. Example:

loop.call_soon_threadsafe(callback, *args)

Almost all asyncio objects are not thread safe, which is typically not a problem unless there is code that works with them from outside of a Task or a callback. If there's a need for such code to call a low-level asyncio API, the loop.call_soon_threadsafe{.interpreted-text role="meth"} method should be used, e.g.:

loop.call_soon_threadsafe(fut.cancel)

To schedule a coroutine object from a different OS thread, the run_coroutine_threadsafe{.interpreted-text role="func"} function should be used. It returns a concurrent.futures.Future{.interpreted-text role="class"} to access the result:

async def coro_func():
     return await asyncio.sleep(1, 42)

# Later in another OS thread:

future = asyncio.run_coroutine_threadsafe(coro_func(), loop)
# Wait for the result:
result = future.result()

To handle signals the event loop must be run in the main thread.

The loop.run_in_executor{.interpreted-text role="meth"} method can be used with a concurrent.futures.ThreadPoolExecutor{.interpreted-text role="class"} or ~concurrent.futures.InterpreterPoolExecutor{.interpreted-text role="class"} to execute blocking code in a different OS thread without blocking the OS thread that the event loop runs in.

There is currently no way to schedule coroutines or callbacks directly from a different process (such as one started with multiprocessing{.interpreted-text role="mod"}). The asyncio-event-loop-methods{.interpreted-text role="ref"} section lists APIs that can read from pipes and watch file descriptors without blocking the event loop. In addition, asyncio's Subprocess <asyncio-subprocess>{.interpreted-text role="ref"} APIs provide a way to start a process and communicate with it from the event loop. Lastly, the aforementioned loop.run_in_executor{.interpreted-text role="meth"} method can also be used with a concurrent.futures.ProcessPoolExecutor{.interpreted-text role="class"} to execute code in a different process.

Running Blocking Code {#asyncio-handle-blocking}

Blocking (CPU-bound) code should not be called directly. For example, if a function performs a CPU-intensive calculation for 1 second, all concurrent asyncio Tasks and IO operations would be delayed by 1 second.

An executor can be used to run a task in a different thread, including in a different interpreter, or even in a different process to avoid blocking the OS thread with the event loop. See the loop.run_in_executor{.interpreted-text role="meth"} method for more details.

Logging {#asyncio-logger}

asyncio uses the logging{.interpreted-text role="mod"} module and all logging is performed via the "asyncio" logger.

The default log level is logging.INFO{.interpreted-text role="py:const"}, which can be easily adjusted:

logging.getLogger("asyncio").setLevel(logging.WARNING)

Network logging can block the event loop. It is recommended to use a separate thread for handling logs or use non-blocking IO. For example, see blocking-handlers{.interpreted-text role="ref"}.

Detect never-awaited coroutines {#asyncio-coroutine-not-scheduled}

When a coroutine function is called, but not awaited (e.g. coro() instead of await coro()) or the coroutine is not scheduled with asyncio.create_task{.interpreted-text role="meth"}, asyncio will emit a RuntimeWarning{.interpreted-text role="exc"}:

import asyncio

async def test():
    print("never scheduled")

async def main():
    test()

asyncio.run(main())

Output:

test.py:7: RuntimeWarning: coroutine 'test' was never awaited
  test()

Output in debug mode:

test.py:7: RuntimeWarning: coroutine 'test' was never awaited
Coroutine created at (most recent call last)
  File "../t.py", line 9, in <module>
    asyncio.run(main(), debug=True)

  < .. >

  File "../t.py", line 7, in main
    test()
  test()

The usual fix is to either await the coroutine or call the asyncio.create_task{.interpreted-text role="meth"} function:

async def main():
    await test()

Detect never-retrieved exceptions

If a Future.set_exception{.interpreted-text role="meth"} is called but the Future object is never awaited on, the exception would never be propagated to the user code. In this case, asyncio would emit a log message when the Future object is garbage collected.

Example of an unhandled exception:

import asyncio

async def bug():
    raise Exception("not consumed")

async def main():
    asyncio.create_task(bug())

asyncio.run(main())

Output:

Task exception was never retrieved
future: <Task finished coro=<bug() done, defined at test.py:3>
  exception=Exception('not consumed')>

Traceback (most recent call last):
  File "test.py", line 4, in bug
    raise Exception("not consumed")
Exception: not consumed

Enable the debug mode <asyncio-debug-mode>{.interpreted-text role="ref"} to get the traceback where the task was created:

asyncio.run(main(), debug=True)

Output in debug mode:

Task exception was never retrieved
future: <Task finished coro=<bug() done, defined at test.py:3>
    exception=Exception('not consumed') created at asyncio/tasks.py:321>

source_traceback: Object created at (most recent call last):
  File "../t.py", line 9, in <module>
    asyncio.run(main(), debug=True)

< .. >

Traceback (most recent call last):
  File "../t.py", line 4, in bug
    raise Exception("not consumed")
Exception: not consumed