Multithreading vs. Multiprocessing in Python | by Gennaro S. Rodrigues | Towards Data Science

Python is often regarded as an almost obscene programming language. A libertarian place where everything is permitted and our wildest dreams come true. Forget about labels and definitions, and, above all, brackets. It is a language that welcomes everyone, from the most experienced programmer (that weird guy that still uses Fortran for some reason you don’t really want to know about) to the egotistic millennial newbie. Well, that is, except if you are working with multiprocessing or multithreading.
The challenges of working with parallelism and concurrency in Python begins with the fact that even the internet does not understand very well how it works. Indeed, I’ve found multiple wrong statements on stackoverflow responses (some with a high count of up votes) and even very serious-looking blogs. Therefore, differently from other cases, where programming may also be called stackoverflowing, you better know at least the basics of what you are doing before starting to work with multiprocessing or multithreading in Python.
Or else, you might have a bad time.
When looking for the difference between python multiprocessing and multithreading, one might have the impression that they work pretty much the same. That could not be more wrong. The key differences are:
A process is an independent instance executed in a processor core. Threads are components of a process and run concurrently (inside that process).
Processes do not share the same memory space, while threads do (their mother’s memory, poetic, right?).
Threads are lighter and cause less overhead. Also, because they share the same memory inside a process, it is easier, faster, and safer to share data.
True parallelism can ONLY be achieved using multiprocessing. That is because only one thread can be executed at a given time inside a process time-space. This is assured by Python’s global interpreter lock (GIL) (see Python GIL at RealPython).
Processes execution is scheduled by the operating system, while threads are scheduled by the GIL.
We should also understand the difference between concurrency and parallelism.
Concurrent execution means that two or more tasks are progressing at the same time.
Parallel execution implies that two or more jobs are being executed simultaneously.
Now remember: multithreading implements concurrency, multiprocessing implements parallelism. Processes run on separate processing nodes.
So how do we decide between multiprocessing and multithreading? The short answer is:
Multithreading for I/O intensive tasks and;
Multiprocessing for CPU intensive tasks (if you have multiple cores available)
But why?
Imagine you have a task that relies on the input from another part of the system, or from the user.
There is no reason to dedicate a process for that job because it would spend most of its time waiting for the stimuli to begin its work. It would be a waste of resources to dedicate a computing node for that, and it would bring no speedup whatsoever, given that the bottleneck is not on execution but on a third-party call. In this case, multithreading is the way to go.
Indeed, multithreading is usually implemented to handle GUIs. E.g., in a text editor program, one thread would be responsible for getting input from the keyboard, while another would check the spelling, and a third one could handle the formatting configurations.
I/O intensive programs can also directly benefit from multithreading because their bottlenecks are usually reading or writing operations (or downloads in the case of web scrappers, for example). Check the following example:
This function loads some URL addresses into memory and writes them in a local text file (50 times, always overwriting the same file). That is pretty dumb, but its purpose is just to stress an I/O bound execution. You can check this code and execute it yourself in this Notebook.
Let us run that method over a list of 16 addresses and check their execution times for some different thread counts. That would get us the following graph (executing on a MacBook Air 2019).
Where the result for one thread stands for the serial execution of the code. We can see by the graphs that multithreading is capable of considerably reducing the algorithm’s execution time. We could keep increasing the number of threads, but the gains would stop once we reach the point where each address is handled by a thread.
Now imagine you got a massive amount of data loaded in memory, and it all needs to be processed. Programs that are computing-bound will benefit from multiprocessing because their bottleneck is time and resources. Image and graphics processing are an excellent example of that. They consist of vast amounts of math operations, which can often be divided into separate tasks because their data is independent. GPUs are the state-of-the-art hardware in this regard, being designed to process large chunks of data in parallel.
To simulate computing-intensive algorithm, we will use the following code:
Again, you can check this code in this notebook and execute it yourself (I strongly suggest you to take a look on that, as we have to make some workarounds to execute it properly on windows).
My MacBook features a dual-core processor with hyper-threading, so there is no use to force more than 4 processes running on it. Those are the results for the performance tests on this benchmark:
The multithreading provided no speedup (the execution times are virtually the same). However, we had a significant speedup when using multiprocessing. It was not enough to drop the execution time to one fourth of the original because of the time spent with process management. Remember that processes are much heavier than threads.
Even though this is not their best use case, there is no reason to expect multiprocessing to perform poorly on I/O intensive tasks. It is a waste of resources: imagine dedicating a processor core to a function that will, for a significant part of its execution, just wait for an input. A processor, even a cluster, only has so many processing nodes. We are better off dedicating them to real parallelism of applications that can only be accelerated by those means.
But for the sake of science, what would happen if we executed the first I/O-bound code using multiprocessing? The following graph shows it.
As expected, multiprocessing is also able to provide speedup on I/O bound computations. It is also noticeable that we kept the speedup even with 8 processes. The reason for that is processes start executing like threads in this case. The speedup is not coming from the parallelism, but from concurrency.
Now, let’s take a deeper look on how we can implement multiprocessing and multithreading in Python and how a Data Scientist can profit from them. There are many different ways to do it, and I encourage you to get some ideas from the documentation (here, here, and here) and try them out on the examples model I provided at this notebook as an exercise.
In a VERY broad aspect, a data science project follows four steps:
Get the data;Process the data;Do something with the data;Store other data
Get the data;
Process the data;
Do something with the data;
Store other data
Now, you may say: “nah, my project is way different”. Well, I beg to differ, but I’m sure you can follow this blog post to get some ideas on how to accelerate your own so-different-from-all-the-others project.
Remember the golden rule: multithreading for I/O bound tasks and multiprocessing for CPU bound task.
Now, it should not be hard to follow that the “get the data” and “store other data” steps are probably the ones where we should use multithreading. All the others (“process the data” and “do something with the data”) are most probably better off using multiprocessing. You may argue that processing a lot of data might present an I/O bottleneck, and in some cases you might be right. If that is your case, you should try to break up this processing into “getting data” and “processing data” parts, so that the best strategy can be applied to each of them.
Let’s see how we can apply what we’ve learned in a classic Natural Language Processing (NLP) problem: text classification. The idea is to find out the category of a news article just by its text (e.g.: if it should be categorised as “sports”, “finances”, “economy”, etc).
If you are here only to see how multiprocessing and multithreading can be used in a real project, just follow along, you don’t really need to know anything about NLP to understand this example. However, if you are interested by the subject, you should start by studying what word vectors are. I like this article. You can also follow this GitHub repository where I and a colleague have been working on some text classification models (there are some cool notebooks there).
Our project is quite simple, we will:
Get some data from Reuters News Agency containing news and their respective labels (defining what type of news they are) (Data source);Vectorize them using GloVe pre-trained word vectors (trained from Wikipedia) (GloVe project page);Train a model using Random Forests with scikit-learn to classify texts under the given labels.
Get some data from Reuters News Agency containing news and their respective labels (defining what type of news they are) (Data source);
Vectorize them using GloVe pre-trained word vectors (trained from Wikipedia) (GloVe project page);
Train a model using Random Forests with scikit-learn to classify texts under the given labels.
By now it shall be straightforward to see that step 1 can possibly be accelerated in Python using multithreading, while step 3 should use multiprocessing.
Let’s start by the pre-trained GloVe word vectors loading. You can check the full code and execute it yourself in this notebook. Loading this pre-trained word vector file can take a lot of time. The file is considerably long, and we have to process it line per line. Each line contains a word and then a list of values for each dimension of its word vector.
The GloveVectorizer() loads the pre-trained vectors in its __init__ function, and can do it both asynchronously or serially. This is how it performs the basic line-per-line file reading:
And here is the multithread implementation:
Again, I strongly recommend you to check how it was implemented in the full code.
The ThreadPoolExecutor runs its threads asynchronously. The last for loop is used to guarantee the execution will only keep going after all the threads submitted to the executor are finished. Check the Python Documentation for more details on how ThreadPoolExecutor works.
But how faster does the vectors loading gets with multithreading? In my MacBook Air the first serial version loads the 400000 word vectors in around 269.19898986816406s. The asynchronous approach loads the 400000 word vectors in 27.559515953063965s, using 10 workers (it could probably reach the same execution time with even less workers, as the bottleneck is reading the lines, not processing).
Now the cool part: training and testing.
Luckly for us, scikit-learn offers multiprocessing nativelly, just by setting it up on the model’s parameters. The two following code examples train a same model with the same data serially or using a set number of jobs (which are mapped to processes in scikit-learn).
Using multiprocessing in scikit-learn is as easy as setting the n_jobs model parameter. Here, we will set it to two:
It’s so easy you may doubt it works. But it does. The following bar graph shows the training time of this same model for different numbers of jobs:
Oh, and if you are curious, the model accuracy (RandomForestClassifier.scoreTrain) turns out not bad at all:
Train score: 0.9992707383773929Test score: 0.9346733668341709
I think it covers pretty much everything. This last example shows how Python multiprocessing and multithreading features can be used to accelerate real projects, and sometimes with little-to-none code modifications. All that glitters is not gold, though. You will soon find out, when looking forward to more complex parallel and asynchronous executions in python that things can get quite messy.