Version Control With Jupyter Notebook | by Shinichi Okada | Towards Data Science

Table of ContentsIntroduction1. Creating a demo repo2. Jupytext setup3. Converting to a python file4. Converting multiple files5. Converted file6. Adding ipynb to .gitignore7. Converting to ipynb files8. Other commands9. Paired notebooksConclusion
Jupyter notebook generates files that contain metadata, source code, formatted text, and rich media. Only one word of change results in thousands of letters in git diff.
Jupytext can save Jupyter Notebook to a git-friendly and human-friendly file format, including Markdown, Python, Julia, Bash, Clojure, Matlab, TypeScript, Javascript, etc.
It also converts these documents into Jupyter Notebooks. In this article, I am going through a step-by-step guide to version control for Jupyter Notebook using Jupytext.
If you are not using Github’s ipynb rendering, Nbviewer or Binder, then Jupytext should be your choice of version control.
Supported extensions are:
levelup.gitconnected.com
towardsdatascience.com
towardsdatascience.com
First, let’s create a new Jupyter Notebook file with the following codes.
x = np.arange(-3, 3, 0.1)y = np.sin(x)plt.plot(x, y)plt.show()
Please create a Github repo.
echo "# jupyter_notebook_version_control" >> README.mdgit initgit add README.mdgit commit -m "first commit"git remote add origin git@github.com:username/jupyter_notebook_version_control.gitgit push -u origin master
I change the word `sin` to `cos` in an ipynb file.
y = np.cos(x)
This link is the result of `git diff`. It generated thousands of letters for three letters.
Let’s install and setup the Jupytext.
pip install jupytext --upgrade
Or for conda users
conda install -c conda-forge jupytext
RESTART Jupyter Notebook.
You can convert an ipynb file to one of the supported files. I will use a python file in this article.
In your terminal, you can run like this.
jupytext --to py <your-file-name>.ipynb
For my case:
jupytext --to py Version_control.ipynb
Outputs:
[jupytext] Reading ./Version_control.ipynb[jupytext] Writing ./Version_control.py
towardsdatascience.com
Let’s convert all ipynb files at once. Please create more files in your directory.
jupytext --to py *.ipynb
Output:
[jupytext] Reading Version_control.ipynb[jupytext] Writing Version_control.py[jupytext] Reading sine.ipynb[jupytext] Writing sine.py[jupytext] Reading tangent.ipynb[jupytext] Writing tangent.py
You can convert a file into a directory. Jupytext will create a new directory if it does not exist.
jupytext --to destination_folder//py *.ipynb
If you prefer you can run jupytext in one of the cells. But this cell will be in your converted file as well.
!jupytext --to py <your-file-name>.ipynb
Let’s see the converted file in your terminal.
cat Version_control.py
My output:
# ---# jupyter:#   jupytext:#     text_representation:#       extension: .py#       format_name: light#       format_version: '1.5'#       jupytext_version: 1.3.3#   kernelspec:#     display_name: Python 3#     language: python#     name: python3# ---x = np.arange(-3, 3, 0.1)y = np.cos(x)plt.plot(x, y)plt.show()
It is very compact and the file size is very small. Nice 😃 👏👏👏👏.
Since we are not tracking ipynb files, we can add it to a .gitignore file. Please create a .gitignore in your project root directory where you have .git directory.
touch .gitignore
Please add *.ipynb and ` .ipynb_checkpoints` to ignore all Jupyter Notebook files. Or add this complete list to your gitignore.
# for Jupytext ignoring ipynb files*.ipynb
At this stage, git will still track changes in .ipynb files. To fix this you need to remove git cache and add all files again.
git rm -r --cached .git add .git commit -m "fixed untracked files"
After changing a line in your Jupyter Notebook to see if .gitignore is working.
# change whatever you wanty = np.arange(-2,2,0.1)
Check it in your terminal:
git status
It should not return a modified file. Let’s run Jupytext one more time to reflect on our change. Please run the following in your terminal.
jupytext --to py Version_control.ipynb
The converted file will be replaced. 😃
[jupytext] Reading ./Version_control.ipynb[jupytext] Writing ./Version_control.py (destination file replaced)
Let’s check the git status.
git statusOn branch masterYour branch is up to date with 'origin/master'.Changes not staged for commit:  (use "git add <file>..." to update what will be committed)  (use "git checkout -- <file>..." to discard changes in working directory)    modified:   Version_control.pyno changes added to commit (use "git add" and/or "git commit -a")
It tracked only the python file, not ipynb. Please run git diff.
git diffdiff --git a/Version_control.py b/Version_control.pyindex 02d91ea..6522717 100644--- a/Version_control.py+++ b/Version_control.py@@ -14,6 +14,7 @@ # --- x = np.arange(-3, 3, 0.1)+y = np.arange(-2,2,0.1) y = np.cos(x) plt.plot(x, y) plt.show()
Please add, commit and push the change.
git add .git commit -m "Update"git push
We are going to clone this repo to another directory and convert it to an ipynb file.
cd ..git clone git@github.com:shinokada/jupyter_notebook_version_control.git my-new-dir
I cloned my repo to a directory called my-new-dir.
cd my-new-dirlsREADME.md    Version_control.py sine.py    tangent.py
Or if you have the tree.
tree.├── README.md├── Version_control.py├── sine.py└── tangent.py0 directories, 4 files
We have all the files we need. Let’s convert it to ipynb file.
From your terminal:
jupytext --to ipynb *.py
Output:
[jupytext] Reading Version_control.py[jupytext] Writing Version_control.ipynb[jupytext] Sync timestamp of 'Version_control.py'[jupytext] Reading sine.py[jupytext] Writing sine.ipynb[jupytext] Reading tangent.py[jupytext] Writing tangent.ipynblsREADME.md      Version_control.py    sine.py    tangent.py.   Version_control.ipynb sine.ipynb   tangent.ipynb
These are other commands you can use.
# convert notebook.md to an .ipynb file and run itjupytext --to notebook --execute notebook.md# update the input cells in the .ipynb file and preserve outputs and metadata jupytext --update --to notebook notebook.py# Turn notebook.ipynb into a paired ipynb/py notebookjupytext --set-formats ipynb,py notebook.ipynb # Update all paired representations of notebook.ipynbjupytext --sync notebook.ipynb
Jupytext can write a given notebook to multiple files. In addition to the original notebook file, Jupytext can save the input cells to a text file — either a script or a Markdown document. Please read more details if you are interested.
Jupytext is easy to use and create human-friendly files which you can edit in another editor as well. If you are using git diff , this is an excellent tool to have. I think this is the most complete open-source tool for version control with Jupyter Notebook at the moment.
Get full access to every story on Medium by becoming a member.
towardsdatascience.com
https://github.com/mwouts/jupytext
https://jupytext.readthedocs.io/en/latest/index.html
https://jupytext.readthedocs.io/en/latest/introduction.html#demo-time
Hello. You made it to the end. Now that you’re here, please help me spread this article. You can also follow me for more Jupyter, Statistics and tech articles.