A Complete Guide to Using Progress Bars in Python | by Chaitanya Baweja | Towards Data Science

We’ve all been there. Our code is running a job and it’s going to take a while. Scratching our head, we pray that our code is not stuck in some infinite cycle. When dealing with large data-sets, even the simplest operations can take hours.
A watched pot never boils. A watched loop never breaks.
If you didn’t guess already, I am an impatient guy. Having an ETA or Progress Bar works wonders for my anxiety.
Fortunately, there are libraries out there that provide this functionality using only a couple of lines. In this tutorial, I will introduce four such libraries and compare them based on a variety of factors.
Code used in this tutorial was tested on an Ubuntu system with Python 3.6.9 installed.
I would highly recommend setting up a virtual environment with all the required libraries for testing. Here is how you can do it.
$ virtualenv progress_env$ source ./progress_env/bin/activate$ pip3 install progress progressbar2 alive-progress tqdm
The four libraries that we are going to work with: progress, progressbar2, alive-progress, tqdm.
Progress is a fairly simple and intuitive library. It provides a wide variety of styles to pick from. All progress bars can be integrated into our loops using either a Context Manager or by wrapping up an iterable object into a method. Let’s take a look at some of them.
A simple progress bar that is filled with hash.
We first import the Bar class from progress.bar module and create its object. We supply the prefix 'Processing...' which will be added to the front of our progress bar. To update the progress bar, we use the next() method at the end of each iteration.
There are multiple arguments available to us like fill and suffix. You can find details regarding them here. Let’s try modifying the fill from # to @ and also look at the ETA.
There are times when we don’t know how long an operation is going to take. In such cases, we can use Spinners to visualize processing rather than progress bars.
Read more about Progress here: Source Code
Python-Progressbar(aka Progressbar2) is an extremely popular and easy to use module. It provides some very powerful features like auto-resizing when the system supports it. You can control the format of the progress-bar in the form of widgets like AbsoluteETA, AdaptiveETA etc.
Let’s take a look at a basic use-case:
Note: Although in the beginning we installed the library as progressbar2, it will be imported as progressbar.
One thing that we didn’t discuss earlier with the progress library: What happens when the for loop contains a print function? Let’s see:
As you can see, this will make the progress bar very messy and sadly, there is no workaround in the progress library for this. However, Progressbar2 allows us to redirect standard output.
Don’t forget to check out all the cool features available in ProgressBar2 here.
If you love animations and you wish to make your progress bars look interactive, then look further. This library contains some of the coolest looking progress-bars available out there. Take a look:
In terms of code, it’s pretty similar. We will present a default progress-bar and then one with a couple modifications:
As you can see above, we get information regarding ETA and iterations per second along with the progress bar. The Github documentation includes details regarding some really powerful features that this library has.
If you need a no-bullshit speedy yet powerful progress-bar, look no further.
TQDM provides a fast framework with a lot of customization and information. Even with all the features, it is still extremely simple to set up. Take a look:
By default, you get info such as percentage, ETA and even iterables per second. TQDM works on all major platforms like Linux, Windows, and Mac. It integrates seamlessly in a console, a GUI, and even in IPython/Jupyter notebooks.
IPython/Jupyter is supported via the tqdm.notebook sub-module. Let’s take an example with nested progress bars:
Note: This feature with multiple progress-bars is unique to TQDM.
There’s also a pandas integration,
Note: TQDM also has an experimental GUI-Version called tqdm-gui.
TQDM is an extremely important project for individuals working in Data Science. You can read more about it here: Documentation.
Progress bars provide us with:
A visual cue that processing is underway.A reliable estimate of the code execution time. (Extremely important when training on large data-sets).Indication if the program is stuck in some error and must be stopped.
A visual cue that processing is underway.
A reliable estimate of the code execution time. (Extremely important when training on large data-sets).
Indication if the program is stuck in some error and must be stopped.
TQDM has the lowest overhead of all the libraries discussed above at about 60ns per iteration. By comparison, the well-established ProgressBar2 has an 800ns/iter overhead.
Most of these libraries don’t have any dependencies.
Progressbar2 and alive-progress officially support print statements along with the progress bar.
In this tutorial, we saw how with only a few lines of code, you can implement progress bars in your python scripts. Hope you found this useful!
How to Use Progress Bars in PythonPython Progress Bar — Stack OverflowProgress Bars for Jupyter Notebook
How to Use Progress Bars in Python
Python Progress Bar — Stack Overflow
Progress Bars for Jupyter Notebook