Quickly label data in Jupyter Lab | by Dennis Bakhuis | Towards Data Science

Asking a random person what a Data Scientist does, he or she might answer that a Data Scientist provides data-driven solutions using state-of-the-art algorithms. If that random person happens to be (or know) a Data Scientist, he or she will most probably respond with the joke: 80% of the time a Data Scientist is cleaning the data and 20% of the time the Data Scientist complains about having to clean the data. Jokes aside, Data Scientists do spend a lot of time cleaning and tuning datasets, and in general are always on a lookout to improve the process. Especially when a task is a bit less exciting such as labeling.
‘I find the process of data labeling very rewarding (and even addictive). Each time I label an example, I feel like my knowledge, experience, and skill are transferred into the model. I sincerely cannot understand when data scientists complain about being asked to do that.’ — Andriy Burkov
For a NLP project we needed a tool for labeling short text into one or more categories. After a short five minutes of ‘lets write it ourselves’, you quickly come to the conclusion that there is probably something already offered in the Python community. Indeed, there are many like Label Studio or Prodi.gy, which are indeed great solutions, but much to general in our taste. Searching a bit longer, we found exactly the tool we had in mind: Pigeon created by Anastasis Germanidis. It was plain simple: in Jupyter you feed a list of text and a list of labels to a function and a widget pops up to walk through each example. The results are given again as a list. The only downside is that it did not support multi-labels yet.
The great thing about sharing code on Github is that everybody can fork it and make changes. Unfortunately, Anastasis did not respond to an email to discuss changes and a pull request. Therefore, we decided to publish the changes as PigeonXT, an extended version of Pigeon.
PigeonXT currently support the following annotation tasks:
binary / multi-class classification
multi-label classification
regression tasks
captioning tasks
Anything that can be displayed on Jupyter (text, images, audio, graphs, etc.) can be displayed by pigeon by providing the appropriate display_fn argument.
Additionally, custom hooks can be attached to each row update (example_process_fn), or when the annotating task is complete(final_process_fn).
As with the original Pigeon, the extended PigeonXT is uploaded to the Python Package Index and can be installed using pip. In this blob post we present a couple of examples. To run these I suggest to first create a new environment. We are managing environment using the Miniconda and the complete procedure is described here. To create a new environment and install the requirements, type the following in a shell:
conda create --name pigeon python=3.7conda activate pigeonconda install nodejspip install numpy pandas jupyterlab ipywidgetsjupyter nbextension enable --py widgetsnbextensionjupyter labextension install @jupyter-widgets/jupyterlab-managerpip install pigeonXT-jupyter
To start Jupyter Lab, just type the following in the same shell:
jupyter lab
All examples can also be found in in the notebook on my Github page.
Multi-class classification, or binary if only two labels are provided is dead easy:
Annotating multiple classes to the same example is called multi-label annotation and is almost identical to the previous example:
Classification of any kind of structure which can interact with Jupyter (images, audio, movies) can also be labeled with the same ease:
While the custom hooks might again add to much complexity, here is an example for a use-case. When having a DataFrame of 1000 examples to label, it might be to much to do in one go. Not only physically in time, but also to prevent from going crazy of boredom. There, you can write a wrapper around the annotation function to select a portion and save the portion to disk. The next time, it checks for the output, and continuous where you have stopped. This part is written in the wrapper and the final_processing function.
As an example for a row-based hook-function, we change the labels to their numerical equivalents and update the DataFrame with each row. You can now see the DataFrame be filled with each step.
There are many useful tools for labeling data. Most of the tools I found were very complete but therefore also relatively complex. We were looking for a simple tool which is usable right from Jupyter. Pigeon was already a great tool but we had to add some functionality for our needs. We also enjoyed the small endeavor of creating/changing a new Python package and sharing it with the community. Please let me know if you have any questions or just find it useful.