How To Easily Merge Multiple Jupyter Notebooks Into One | by Amal Hasni | Towards Data Science

Written by: Amal Hasni & Dhia Hmila
Juypyter Notebooks are very essential tools for Data Scientists. They offer multiple practical options for interactive computing as they combine code, text, and visualizations in a single document. It is common to choose to use multiple separate Notebooks in a single project for organizational purposes. The problem is when a manager or a client asks for a quick demo and you need to merge your different Notebooks quickly, reorganizing cells can be a long tedious sequence of copy-paste.
Since Jupyter’s interface doesn’t make it easy, we thought It’s time to create our own solution. In this article, we will show you how to reorganize and concatenate two notebooks in a time-efficient way. What you’ll learn will allow you to reorganize, filter, change the notebook’s cell using python code.
Table Of Contents:· Getting to know Notebooks’s structure· Concatenating Notebooks· Going further with nbformat
I don’t know if you’ve ever tried to open a Jupyter notebook file (that has .ipynb as an extension) with a text editor. If you did, you’ve either seen weird gibberish or you’ve recognized the JSON format.
If you don’t know what the JSON format is, it stands for JavaScript Object Notation and is a way to store objects/data in a human-readable way. It’s quite neat and simple (you can learn more about this here ).
So like I’ve said IPYNB files are stored in the JSON plain text format and if you open one up, it will look like this:
As you can see, it’s a well-structured dictionary containing:
metadata: basically, a dictionary containing information about the kernel and language that are used ( and more ).
nbformat and nbformat_minor: These are just the notebook format version (here it’s 4.0)
cells: This is what we’ll, most likely, be interested in and it contains a list of the notebook cells.
Each cell is represented by a similar dictionary containing different key-value pairs:
You can see in this screenshot where each field goes:
Now you know enough to start playing with the notebook’s cells. But, if you want to go more into details, you can check out the documentation at this link.
So the practical example we chose is merging two notebooks together. It’s a fairly simple example but you’ll get to see how to read, write and tweak the notebooks, and depending on your use case, you can adapt the code to your needs.
If you want, you can download example notebooks to try the code at this link.
So let’s start by implementing a function to read IPYNB files. We’ll use the json module included in python's standard library:
Now reading the files only takes two lines of code:
first_notebook = read_ipynb('first_notebook.ipynb')second_notebook = read_ipynb('second_notebook.ipynb')
Though we probably don’t necessarily need to copy the notebook in this example, it may come in handy if you want to play around with the notebooks. Here, again, we’ll use the copy module of the standard library:
import copyfinal_notebook = copy.deepcopy(first_notebook)
So here comes the part where we actually merge the cells:
final_notebook['cells'] = first_notebook['cells'] + second_notebook['cells']
And finally, let’s write a helper function to export the notebook into the Jupyter Notebook format and export our final_notebook with it:
The Jupyter Development Team gave us the package nbformat to make similar operations such as reading ipynb files or creating new code cells with nbformat.v4.new_code_cell .
To have an example, let’s reproduce the equivalent code to concatenate two notebooks with nbformat:
If you want a more exhaustive list of functions provided by nbformat, you can check out their documentation in the following link.
Taking a deeper look at the structure of a Jupyter Notebook gives the necessary knowledge to create solutions for cell manipulation. This article details two solutions that give you a workaround for the tedious copy-pasting you’ll normally need to merge two Notebooks into one. You’ll find all the code used in this article in this Github repository.
Depending on your use case, you can go further and create scripts to make other modifications you need in an automatic, time-efficient way.
We hope you found this article useful. Thank you for sticking around this far, stay safe and we will see you in our next article 😊!