A better way for asynchronous programming: asyncio over multi-threading | by Qian (Aria) Li | Towards Data Science

If you’ve come here, it is probably because you have heard of asyncio module that’s introduced in Python 3.4. You might have been wondering whether you should begin using it. I remember the moment I got stunned by its brilliant performance. Before I learned about asyncio, I was using multi-processing and threading to speed jobs up. They worked well until one day I hit a bottleneck handling hundreds of millions of URL calls. The challenge pushed me to look for other feasible solutions to the I/O bound task. Though its syntax and concept seem complex at first, asyncio turns out to be a life-changer.
Asyncio is all about asynchronous programming, but it is not the only option that runs jobs asynchronously. You might pause me here and ask, what’s asynchronous programming? In simple words, asynchronous programming allows different tasks to start execution without waiting for the previous one to be completed. This is huge because the vast portion of the time taken to read the URLs is due to the network delay. It’s unnecessary to suffer from high latencies and keep the machines idle while waiting for a response from another server. Asynchronous programming solves the exact problem.
Multi-threading is a traditional solution that performs tasks asynchronously. Both asyncio and multi-threading run concurrently. Oh, wait, what’s concurrency? Concurrency is a concept that opposes to parallelism; it means executing multiple tasks at the same time but not necessarily simultaneously while parallelism means executing tasks simultaneously. The difference between these two concepts is not important in the blog (see here for more about the difference), but keep in mind that we are not exploring parallelism solution in this blog. As good as it sounds, parallelism is not ideal for I/O bound tasks; it works well with CPU bound job.
So why asyncio is faster than multi-threading if they both belong to asynchronous programming? It’s because asyncio is more robust with task scheduling and provides the user with full control of code execution. You can pause the code by using the await keyword and during the wait, you could run nothing or go ahead executing other code. As a result, resources are not locked down during the wait.
The way the tasks take turns for multi-threading is completely different. In threading, the Python interpreter is responsible for task scheduling. Having no prior knowledge of the code or the tasks, the interpreter gives each thread a slice of time to utilize the resources in turns before switching to the next thread. There is some level of inefficiency in this line of arrangement. A simple task could be cut off in the middle no matter how trivial it is. Resources could be locked down to a thread where the task is still waiting for a response from an outside server and therefore not ready to proceed. Consequently, there is still some waste in both time and resources.
Before we get to asyncio, let’s use multi-threading as a benchmark! My machine has 8 CPUs for all the examples.
All libraries for the test are imported in the front. Yes, I use logging. I’ve basically dropped print and switched to logging. And remember to always use exceptions. A detailed exception is best. But if you don’t know or care about it, use general Exception!
There are two functions in the code snippet.
The first function, fetch_url uses requests module and GET method to retrieve data from a specified URL. The try and except block catch server or timeout error so we would run multi-threading without interruption.
The second function, fetch_all uses a pool of threads to execute the first function asynchronously via ThreadPoolExecutor() from the concurrent.futures module. The map method gathers responses from all threads.
Outside the two functions, it calls fetch_all several times. Each time, we pass a URL list of a different length and record the time it takes.
Now let’s look at some performance results:
INFO:root:Fetch 1 urls takes 0.7269587516784668 secondsINFO:root:Fetch 10 urls takes 0.7232849597930908 secondsINFO:root:Fetch 100 urls takes 5.631572008132935 secondsINFO:root:Fetch 500 urls takes 10.897085905075073 secondsINFO:root:Fetch 1000 urls takes 20.450702905654907 seconds
The result is not bad at all. You can see that multi-threading has done a decent job reading 1000 URLs.
import asynciofrom aiohttp import ClientSessionasync def fetch(url):    async with ClientSession() as session:        async with session.get(url) as response:            return await response.read()
This is basically asyncio version of fetch_url. I use aiohttp because it provides an excellent client session where we can make HTTP requests asynchronously.
Besides aiohttp.ClientSession, the code probably looks strange with async and await syntax. These two are the key symbols of asyncio. The async def and async with statement create coroutine objects whose execution can be suspended; the await keyword tells the program which execution to wait for. The use of the two keywords makes it possible for the code to run asynchronously.
We don’t directly call the fetch function to fire up asynchronous execution. Instead, we need to create an event loop and add tasks in the loop. The below two lines of code help you get started with fetching only one URL.
loop = asyncio.get_event_loop()loop.run_until_complete(fetch(url))
To run multiple URLs and asynchronously gather all responses, you would need to utilize ensure_future and gather functions from asyncio.
I hope you still remember the previous multi-threading example because I’m presenting you with a complete asyncio version! The code snippet has the same structure as the multi-threading example.
It has an upgraded version of fetch function from the previous asyncio introduction. It’s upgraded by enabling additional exceptions. Another big change is that the client session has become an argument of the function.
The fetch_async is an asyncio version of fetch_all. The function has created a client session and inside it, an event loop is created and loaded with tasks, where one URL request is one task. It’s an important lesson learned that having one client session for all your HTTP requests is a preferred and faster way.
The result of the asynchronous operation is a future object and a gather method is used in fetch_async to aggregate the results.
A timer is wrapped around the fetch_async function call to track time.
The code was executed on the same 8-CPU machine as before and results are displayed below:
INFO:root:Fetch 1 urls takes 1.5993337631225586 secondsINFO:root:Fetch 10 urls takes 1.2335288524627686 secondsINFO:root:Fetch 100 urls takes 2.6485719680786133 secondsINFO:root:Fetch 500 urls takes 4.8839111328125 secondsINFO:root:Fetch 1000 urls takes 6.814634084701538 seconds
You might have noticed that in both examples, the time it takes for 1 URL is slightly longer than the time for 10 URLs. This might look contrary to common practice. But think about it. Both multi-threading and asyncio are meant for “parallel” jobs, and they both have fired up extra resources to complete the jobs faster. Using either of them to read one URL won’t save your time if not waste your time.
Between 1 to 10 URLs, asyncio takes more time in seconds to send requests and gather responses. It could mean that multi-threading is preferred for small I/O bound tasks.
From somewhere between 10 to 100 URLs, the execution time of asyncio drops under that of multi-threading. With the increase in the number of URLs, the time difference between the two solutions is getting larger. That concludes the superior performance of asyncio on a large load of URL requests.