Stop Waiting! Start using Async and Await! | by Simon Hawe | Towards Data Science

The most important ingredient for us data scientists is data. How do we get the data we need into our programs? We do that through some I/O-operations like querying a database, loading files from our disk, or downloading data from the web through HTTP requests. These I/O-operations can take quite some time where we just hang around waiting for the data to be accessible. This gets even worse when we have to load multiple files, do multiple database queries, or perform multiple HTTP requests. Most often, we perform these operations sequentially, which leads to executing 100 I/O-operations in total taking 100 times longer than executing a single operation. Now waiting is not only annoying but becomes a real pain. But wait, not too long :), does it make sense to wait for the response to a request before we fire up another completely independent request? Or as a daily life example, when you write an email to two persons, would you wait to send the email to the second person until you’ve received a response from the first one? I guess not. In this article, I want to show you how to significantly reduce waiting time for IO-bound problems using the Asynchronous IO, short AsyncIO, programming paradigm in Python. I will not dive too much into technical details, but rather keep it fairly basic and show you a small code example, which hopefully facilitates your understanding.
Assume we want to download three different files from a server. If we do that sequentially, most of the time our CPU is idle waiting for the server to respond. If the response time dominates, the total execution time is given as the sum over the individual response times. Schematically, this looks like shown in Picture 1.
I think an idle CPU sounds suboptimal. Wouldn’t it be better to send the requests sequentially, be idle until all those requests have finished and combine all the responses? In that scenario, again assuming the response time to dominate, the total execution time is roughly given as the maximum response time out of all requests. Schematically this is shown in Picture 2.
Note that the blue bar just visualizes the time between a request being sent and the response is available. Instead of just having an idle CPU in that timeframe, we should make use of it and perform some work. That’s exactly what we are doing here. When a response from one request arrives we already process it while still waiting for the other responses to arrive. That sounds great, doesn’t it? So how can we write code that makes use of that? As already spoiled in the introduction, we can use AsyncIO to that end.
Asynchronous is a higher-level programming paradigm, where you start a task, and while you don’t have the result of that task, you do some other work. With this, AsyncIO gives a feeling of concurrency despite using a single thread in a single process. The magic ingredients to make that work are an event loop, coroutines, and awaitable objects which are coroutine-objects, Tasks, and Futures. Just very briefly
An event loop orchestrates the execution and communication of awaitable objects. Without an event loop, we cannot use awaitables, hence, every AsyncIO program has at least one event loop.
A native coroutine is a python function defined with async def. You can think of it as a pausible function that can hold state and resume execution from the paused state. You pause a coroutine by calling await on an awaitable. By pausing, it releases the flow of control back to the event loop, which enables other work to be done. When the result of the awaitable is ready, the event loop gives back the flow of control to the coroutine. Calling a coroutine returns a coroutine-object, that must be awaited to get the actual result. Finally, it’s important to remember that you can call await only inside a coroutine. Your root coroutine needs to be scheduled on the event loop by a blocking call.
Below you find some very basic code that hopefully facilitates your understanding
import asyncioasync def sample_coroutine():    return 1212async def main_coroutine():    coroutine_object = sample_coroutine()    # With await, we stop execution, give control back to the     # eventloop, and come back when the result of the     # coroutine_object is available.    result = await coroutine_object    assert result == 1212# Blocking call to get the event loop, schedule a task, and close# the event loopasyncio.run(main_coroutine())
If you are interested in more comprehensive introductions, I refer you to this, this, and that.
Enough theory, let’s try it out. Prerequisite to use native coroutines is Python 3.5+, however, asyncio is evolving rapidly so I would suggest using the most recent Python version which was 3.7+ at the time of writing. As an example, I download a couple of dog images. For comparison, I do the same operations in both synchronous, and asynchronous fashion. To execute HTTP requests, I use Requests for the synchronous part and AIOHTTP for the async part. I have intentionally left out any kind of error checking to avoid convoluting the code.
So let’s first install the necessary modules
pip install aiohttp requests
Next, I import the modules and add a small helper function
import asyncioimport timefrom typing import Any, Iterable, List, Tuple, Callableimport osimport aiohttpimport requestsdef image_name_from_url(url: str) -> str:    return url.split("/")[-1]
Now, given a bunch of image URLs, we can sequentially download and store them as a List of pairs (image-name, byte array) by
def download_all(urls: Iterable[str]) -> List[Tuple[str, bytes]]:    def download(url: str) -> Tuple[str, bytes]:        print(f"Start downloading {url}")        with requests.Session() as s:            resp = s.get(url)            out= image_name_from_url(url), resp.content        print(f"Done downloading {url}")        return outreturn [download(url) for url in urls]
I have added a few print statements such that you see what happens when executing the function. So far so good, nothing new up until now, but here comes the async version
async def donwload_aio(urls:Iterable[str])->List[Tuple[str, bytes]]:    async def download(url: str) -> Tuple[str, bytes]:        print(f"Start downloading {url}")        async with aiohttp.ClientSession() as s:            resp = await s.get(url)            out = image_name_from_url(url), await resp.read()        print(f"Done downloading {url}")        return out            return await asyncio.gather(*[download(url) for url in urls])
Ah ha, this looks almost identical, apart from all those async and await keywords. Let me explain to you what’s happening here.
download_aio is a coroutine as it is defined with async def. It has to be a coroutine because we call other coroutines within it.
In the download coroutine, we create a Session object using an async context manager (async with) and await the result of the get request. At this point, we perform the potentially long-lasting HTTP request. Through await we pause execution and give other tasks the chance to work.
asyncio.gather is probably the most important part here. It executes a sequence of awaitable objects and returns a list of the gathered results. With this function, you can achieve a feeling of concurrency as shown in Picture 2. You cannot schedule too many coroutines with gather, it’s in the order of a few hundreds. When you run into issues with that, you can still partition the calls into smaller sized chunks that you pass to gather one by one. Like calling a coroutine returning an awaitable, calling gather also returns an awaitable that you have to await to get the results.
Let’s put that together, run it, and compare how long it takes to download the doggys.
Below you find the code to perform both the synchronous and asynchronous Http calls.
if __name__ == "__main__":    # Get list of images from dogs API    URL = "https://dog.ceo/api/breed/hound/images"    images = requests.get(URL).json()["message"]# Take only 200 images to not run into issues with gather         reduced = images[:200]    st = time.time()    images_s = download_all(reduced)    print(f"Synchronous exec took {time.time() - st} seconds")st = time.time()    images_a = asyncio.run(donwload_aio(reduced))    print(f"Asynchronous exec took {time.time() - st} seconds")
In a bit simplified version, the synchronous version prints out
Start 1, End 1, Start 2, End 2, ... , Start 200, End 200
which reflects the flow shown in Picture 1. The async counterpart prints out something like
Start 1, Start 2, ..., Start 200, End 3, End 1, ..., End 199
which reflects what is shown in Picture 2. Awesome, that is what I have promised you. But I think that alone is not very convincing. So, save the best for last and let’s look at the execution times. To download 200 images on my machine, the synchronous call took 52.7 seconds, while the async one took 6.5 seconds which is about 8 times faster. I like that! The speedup varies based on the maximum download time of an individual item, which depends on the item’s size and the load the server can handle without slowing down.
Use AsnycIO for IO-bound problems that you want to accelerate. There are tones of modules for IO-bound operations like Aioredis, Aiokafka, or Aiomysql just to mention a few. For a comprehensive list of higher-level async APIs visit awesome-asyncio.
You can only await a coroutine inside a coroutine.
You need to schedule your async program or the “root” coroutine by calling asyncio.run in python 3.7+ or asyncio.get_event_loop().run_until_complete in python 3.5–3.6.
Last but most important: Don’t wait, await!
Hopefully, you’ve learned something new and can reduce waiting time. Thank you for following along and feel free to contact me for questions, comments, or suggestions.