# Asynchronous Programming in Python

## Summary


Asynchronous programming is a programming paradigm that enables better concurrency, that is, multiple threads running concurrently. In Python, `asyncio` module provides this capability. Multiple **tasks** can run concurrently on a single thread, which is scheduled on a single CPU core.

Although Python supports multithreading, concurrency is limited by the Global Interpreter Lock (GIL). The GIL ensured that only one thread can acquire the lock at a time. Asynchronous programming doesn't solve the GIL limitation but still enables better concurrency. 

With multiprocessing, task scheduling is done by the operating system. With multithreading, the Python interpreter does the scheduling. In Python's asynchronous programming, scheduling is done by what's called **the event loop**. Developers can specify in their code when a task voluntarily gives up the CPU so that the event loop can schedule another task. For this reason, this is also called **cooperative multitasking**.

## Discussion

### What's the background for adopting asynchronous programming in Python?

Python has long supported both multiprocessing and multithreading. **Multiprocessing** can make use of multiple CPU cores but at the overhead cost of inter-process communication (IPC). Each process has its own Python interpreter and GIL. **Multithreading** avoids the IPC overhead by having all threads share the same memory space. But scheduling these threads is limited by the GIL. 

For **I/O-bound** threads, if a thread waits on I/O, GIL is automatically released and given to another thread. For **CPU-bound** threads, GIL doesn't give us a mechanism to run all threads concurrently, even if the system has multiple CPU cores. In either case, asynchronous programming helps us achieve better concurrency (for the single CPU core scenario). 

GIL is used in Python's default CPython implementation. CPython's use of GIL makes memory management thread safe. Other implementations such as JPython and IronPython are not limited by the GIL. They solve memory management and thread synchronization within their virtual machines (JVM/CLR). Historically, CPython's GIL made it easy to interface with C extensions in a thread-safe manner, which made Python popular. 


### Which are the basic constructs that enable asynchronous programming in Python?

Executing asynchronous code requires an **event loop**. Python provides a default event loop implementation. It's also possible to use alternative implementations such as `uvloop`, which has shown at least 2x better performance. 

The event loop executes **coroutines**, one at a time. A coroutine is simply a method or function defined with `async` keyword. A coroutine needs to be added to the event loop, such as using `asyncio.run()`. 

When a coroutine waits for the result of another coroutine using the `await` keyword, it gets suspended. The event loop then schedules another coroutine that's ready to run. More formally, a coroutine waits on an **awaitable object**. This can be another coroutine, a Task or a Future. 

A **Task** is really a wrapper on a coroutine typically created with `asyncio.create_task()`. Via the Task, the coroutine is automatically scheduled. 

A **Future** is a low-level object that represents the eventual result of an asynchronous operation. When a Future is awaited, it means the coroutine will wait until the Future is resolved in some other place. 


### Which are some basic API calls worth knowing for a beginner?

We can create multiple event loops in a thread but only one can be active at a time. Relevant APIs to learn are `asyncio.new_event_loop()`, `asyncio.set_event_loop()`, `asyncio.get_event_loop()` and `asyncio.get_running_loop()`. If `asyncio.get_event_loop()` is called without a prior call to `asyncio.set_event_loop()`, a new event loop is automatically created and set as the current one. 

Event loop methods include `run_until_complete()`, `run_forever()`, `is_running()`, `is_closed()`, `stop()` and `close()`. Callbacks can be scheduled with `call_soon()`, `call_soon_theadsafe()`, `call_later()` and `call_at()`. Another useful method of loop is `create_task()`, which is also possible with `asyncio.create_task()`.  

Loop also includes methods to manage network connections and entities: `create_connection()`, `create_datagram_endpoint()`, `create_server()`, `sendfile()`, `start_tls()`, plus low-level methods to work directly with sockets. 

`asyncio.run()` runs a coroutine, in the process creating an event loop and closing it at the end. This can't be called if an event loop is already running in the current thread. To run many tasks concurrently, call `asyncio.gather()`. Other useful methods include `asyncio.wait()`, `asyncio.wait_for()`, `asyncio.current_task()` and `asyncio.all_tasks()`. To sleep asynchronously, use `asyncio.sleep()` rather than `time.sleep()`. 


### What some essential tips when working with asynchronous Python code?

An event loop can run in any thread but it must run in the main thread if it has to handle signals and execute subprocesses. 

To schedule callbacks from another thread, use `loop.call_soon_threadsafe()` rather than `loop.call_soon()`. 

Note that calling a coroutine function doesn't actually execute the function and return the result. It only returns a coroutine object, which must be passed into `asyncio.run()`. 

Avoid CPU-bound blocking calls. For example, if a CPU-intensive computation takes 1 second, all asynchronous tasks and I/O operations will be delayed by 1 second. Use `loop.run_in_executor()` along with `concurrent.futures.ThreadPoolExecutor` to execute blocking code in a different thread or even a different process. 


### Which Python packages or modules enable asynchronous programming?

Module `asyncio` is the main one for asynchronous programming in Python. While `gevent` and `eventlet` achieve similar behaviour, `asyncio` is easier and more approachable even for non-experts. 

Use module `threading` for I/O-bound concurrent operations. Use `multiprocessing` for CPU-bound parallel computations. Equivalently, `concurrent.futures.ThreadPoolExecutor` and `concurrent.futures.ProcessPoolExecutor` can be used as these provide a simpler API.  

Among the alternatives to `asyncio` are `curio` and `trio`. Both these are somewhat compatible with `asyncio`. Trio in particular claims to focus on usability and correctness. 

Timo Furrer curates a list of asynchronous Python frameworks and packages. Among the web frameworks are aiohttp, Tornado, Sanic, Vibora, Quart and FastAPI. For message queues we have aioamqp, pyzmq, and aiokafka. For testing we have aiomock, asynctest, and pytest-asyncio. For databases we have asyncpg, aioredis, and aiomysql. The popular `requests` library with asynchronous support is called `requests-async`. 

To make use of multicores or even multiple machines, Apache Spark (via PySpark) should be considered. Meanwhile, work is going on to bring multicore support within the interpreter via projects Software Transactional Memory, Dask and PyParallel. 


### How can I debug asynchronous Python code?

By default, `asyncio` runs in production mode. There are many ways to run it in debug mode: 

    + Set `PYTHONASYNCIODEBUG` environment variable to 1
    + Use `-X dev` command line option
    + Use argument `debug=True` to `asyncio.run()` (default is `debug=False`)
    + Call `loop.set_debug(True)` (can inspect current value with `loop.get_debug()`)Adjust the logging level by calling for example `logging.getLogger("asyncio").setLevel(logging.WARNING)`. Python interpreter will emit log messages for never-awaited coroutines or never-retrieved exceptions. In debug mode, we get more information about these issues. 

The third-party library aiodebug might help in debugging asyncio programs. 


### How's the performance of asynchronous programming in Python?

Performance of `asyncio` falls short of the what Node.js and Go can achieve. However, when `asyncio` is used along with `uvloop` (for the event loop) and `httptools` (for HTTP), it gives best performance in terms of both throughput and response time. 

While `aiohttp` offers asynchronous HTTP, it has a slow HTTP parser. This limitation is addressed by `httptools`. However, in one experiment, `aiotools` outperformed its synchronous equivalents such as Flask or Django. It was 26x faster than Flask. 

In another experiment involving multiple HTTP calls, total runtime was 13.1 secs (synchronous, 1 thread), 1.7 secs (synchronous, 20 threads), and 1.3 secs (aiohttp, 1 thread). When used with Gunicorn web server, `aiohttp` was 2x faster than Flask when a single Gunicorn worker was used. 

Another experiment noted that it's unfair to compare synchronous and asynchronous web frameworks without adjusting the **number of workers**. Synchronous frameworks must be given more workers since the entire worker blocks on I/O. With more workers, they can utilize all the CPU cores. This experiment showed that synchronous frameworks did better. 

## Milestones

Oct  
2002

First version of **Twisted** is released. At a time when the Python standard doesn't offer asynchronous support out of the box, Twisted supports it. It's a Python-based event-driven framework useful for building networked clients and servers. It supports multiple protocols and interfaces. 

2009

Ryan Dahl and others at Joyent develop **Node.js**. Asynchronous programming is at the heart of Node.js. With subsequent widespread adoption of Node.js, it becomes important for Python to start looking into asynchronous programming. What Node.js calls promises, Python calls them **awaitables**. 

Dec  
2012

Work towards standardizing asynchronous support in Python begins as part of **PEP 3156**, titled *Asynchronous IO Support Rebooted: the "asyncio" Module*. This supersedes an earlier proposal (PEP 3153). It also acknowledges earlier work of Twisted, Tornado, and ZeroMQ that have supported asynchronous calls in their packages. 

Oct  
2013

Version 0.1.1 of `asyncio` is released on PyPI. Version 0.4.1 comes out in April 2014. 

Mar  
2014

**Python 3.4** is released. Module `asyncio` is formally introduced as part of this release. 

Nov  
2015

Python 3.5 is released with support for **awaitable objects**, **coroutine functions**, **asynchronous iteration**, and **asynchronous context managers**. This includes syntaxes `async def`, `async for`, `async with` and `await`. Any object that has the `&lowbar;&lowbar;await()&lowbar;&lowbar;` method can be awaited. These changes are expected to make asynchronous programming a lot easier. The async/await syntax of this release is inspired by `yield from` of Python 3.3 and `asyncio` module of Python 3.4. 

Dec  
2016

Python 3.6 is released. **Asynchronous generators** and **asynchronous comprehensions** are now supported. The relevant proposals for these are PEP 525 and PEP 530 respectively. 

Jun  
2018

Python 3.7 is released in which `async` and `await` are now **keywords**. This release improves the `asyncio` module in terms of **usability** and **performance**. For example, assuming `main()` is our coroutine, scheduling a coroutine to the event loop is as simple as calling `asyncio.run(main())`. This simplifies the earlier syntax of `loop = asyncio.get_event_loop()` and `loop.run_until_complete(main())`. 

Oct  
2019

Python 3.8 is released with asynchronous support for unit testing. Specifically, `AsyncMock` is added along with suitable assert functions. In the `unittest` module, coroutines can be used as test cases with `unittest.IsolatedAsyncioTestCase`.