Fifteen Jupyter Notebook Extensions to a Docker Image | by Bruce H. Cottman, Ph.D. | Towards Data Science

The Docker image for our Jupyter Python and R users required them to set their Nbextensions preferences after every launch. We were able to increase Jupyter notebook user productivity by setting commonly used Nbextensions preferences in the Docker image.
We have a shared disk where anybody can copy the template files to their local directory structure.
|-- dockerSeasons     |-- dev         |--- Dockerfile         |--- docker-compose.yml     |-- test         |--- Dockerfile         |--- docker-compose.yml
Each member of Dev and Test copies Release 1.3.0dockerSeasons directory over to their PROJECTS directory.
- PROJECTS...    |-- <project-name>        |-- dockerSeasons             |-- dev                 |--- Dockerfile                 |--- docker-compose.yml                 |--- README.md                 |--- <ln> requirements.txt                          |-- test                 |--- Dockerfile                 |--- docker-compose.yml                 |--- README.md                 |--- <ln> requirements.txt        |-- src        |-- test        |--- requirements.txt        |--- README.md..
Note: Dev 1.0.3 is able to use the 0.0.3 PROJECTS directory structure unchanged. Release 3.0.1dockerSeasons overwrites docker 0.0.3 directory.
Test modifies <project-name>/requirements.txt for the difference in Python packages they do and don't require. Then the Test symbolically links <project-name>/requirements.txt(in the <project-name>/dockerSeasons/test directory) with the command:
# MacOS and linux variants$ ln ../../requirements.txt requirements.txt
The only differences in the Test directory are the files requirements.txt, the files in directory dockerSeasons/test/ . Test moves this directory structure to stage server and 3 redundant production servers. The server farm is internal.
1.0.3 released a dockerfile template that enabled Dev and Test to customize each Docker image by requirements.txt.
In a previous article, I showed the Dockerfile used to create a Docker image with Jupyter notebook. We are now on DES 1.3.0 (Docker Enterprise Solution) with a different specification of the Dockerfile for Jupyter notebook users.
The meaningful change is that jupyterthemes (nbextentsions) stage after requirements.txt. Through trial and feedback, we found that Dev and Test changed requirements.txt less often than jupyterthemes.
Changing the order of staging from less frequent changes to more frequent changes allowed us to take advantage of Docker buffering. The result is faster Dockerfile builds than prior.
Depending on what is in requirements.txt, we experience full builds up to 10 minutes. By putting jupyterthemes last, we experience builds (no requirements.txt change) of approximately 30 seconds.
You might consider the frequency of changes and order of builds in your Dockerfile builds.
The DES 1.3.0 release of the partial Dockerfile is:
FROM python:3.7FROM jupyter/minimal-notebookWORKDIR $HOMERUN python -m pip install --upgrade pipCOPY requirements.txt ./requirements.txtRUN python -m pip  install -r requirements.txRUN python -m pip install --upgrade --no-deps --force-reinstall notebook#RUN python -m pip install jupyterthemesRUN python -m pip install --upgrade jupyterthemesRUN python -m pip install jupyter_contrib_nbextensions
Note: Do not execute the following commands from your terminal shell. If you do, you install jupyterthemes in your local copy of jupyter. The bigger problem is that you reset your local nbextentsions settings in your local copy of jupyter.
pip install jupyterthemespip install --upgrade jupyterthemespip install jupyter_contrib_nbextensions
We had to build the Dockerfile in two steps because we need to find what jupyterthemes are available and what their names are. First, we executed the partial Dockerfile listed above.
We launched juptyer using command updev (described later in this article). We then ran the following in a Jupyter notebook.
!jupyter nbextension list
output->>
Known nbextensions:  config dir: /home/jovyan/.jupyter/nbconfig    notebook section      nbextensions_configurator/config_menu/main  enabled       - Validating: problems found:        - require?  X nbextensions_configurator/config_menu/main      contrib_nbextensions_help_item/main  enabled       - Validating: OK      jupyter-js-widgets/extension  enabled       - Validating: problems found:        - require?  X jupyter-js-widgets/extension      jupyter-notebook-gist/notebook-extension  enabled       - Validating: problems found:        - require?  X jupyter-notebook-gist/notebook-extension      autosavetime/main  enabled       - Validating: OK      codefolding/main  enabled       - Validating: OK      code_font_size/code_font_size  enabled       - Validating: OK      code_prettify/code_prettify  enabled       - Validating: OK      collapsible_headings/main  enabled       - Validating: OK      comment-uncomment/main  enabled       - Validating: OK      equation-numbering/main  enabled       - Validating: OK      execute_time/ExecuteTime  enabled       - Validating: OK      gist_it/main  enabled       - Validating: OK      hide_input/main  enabled       - Validating: OK      spellchecker/main  enabled       - Validating: OK      toc2/main  enabled       - Validating: OK      toggle_all_line_numbers/main  enabled       - Validating: OK    tree section      nbextensions_configurator/tree_tab/main  enabled       - Validating: problems found:        - require?  X nbextensions_configurator/tree_tab/main  config dir: /opt/conda/etc/jupyter/nbconfig    notebook section      plotlywidget/extension  enabled       - Validating: OK
Now we had the Nbextensions names to add to the 1.3.0 release dockerfile!
The dockerfile for Dev enabled the following Nbextensions shown after the comment #enable the Nbextensions.
FROM python:3.7FROM jupyter/minimal-notebookWORKDIR $HOMERUN python -m pip install --upgrade pipCOPY requirements.txt ./requirements.txtRUN python -m pip  install -r requirements.txRUN python -m pip install --upgrade --no-deps --force-reinstall notebook#RUN python -m pip install jupyterthemesRUN python -m pip install --upgrade jupyterthemesRUN python -m pip install jupyter_contrib_nbextensionsRUN jupyter contrib nbextension install --user# enable the NbextensionsRUN jupyter nbextension enable contrib_nbextensions_help_item/mainRUN jupyter nbextension enable autosavetime/mainRUN jupyter nbextension enable codefolding/mainRUN jupyter nbextension enable code_font_size/code_font_sizeRUN jupyter nbextension enable code_prettify/code_prettifyRUN jupyter nbextension enable collapsible_headings/mainRUN jupyter nbextension enable comment-uncomment/mainRUN jupyter nbextension enable equation-numbering/mainRUN jupyter nbextension enable execute_time/ExecuteTime RUN jupyter nbextension enable gist_it/main RUN jupyter nbextension enable hide_input/main RUN jupyter nbextension enable spellchecker/mainRUN jupyter nbextension enable toc2/mainRUN jupyter nbextension enable toggle_all_line_numbers/main
Docker-Compose is used to manage several containers at the same time for the same application. This tool offers the same features as Docker but allows you to have more complex applications.
Docker-Compose can merge more than one Docker container into one runtime image. Currently, we use Docker-Compose for only one Docker image (dockerfile.yaml).
version: '3'services:  dev:    build: '.'    ports:        - "127.0.0.1:8889:8888"    volumes:      - ../../../.:/docker    working_dir: /docker
note: When you use command updev (described later in this article) the docker-compose command volume: - ./../../. causes <path-to-projects> to be mapped to /docker , the internal directory of the docker image. The subsequent launch of jupyter uses <path-to-projects> as its top level directory. Please use the example directory structure shown above or else substitute your own local directory structure for - ./../../. .
The Docker commands are setup by adding the following commands to your  ̃/.bashrc_profile or  ̃/bashrc.txt.
devdir='<path-to-projects>/photon/photonai/dockerSeasons/dev/'testdir='<path-to-projects>/photon/photonai/dockerSeasons/test/'echo $devdirecho $testdirexport testdirexport devdir#alias updev="(cd $devdir; docker-compose up) &"alias downdev="(cd $devdir; docker-compose down) &"alias builddev="(cd $devdir; docker-compose build) &"#alias uptest="(cd $testdir; docker-compose up) & "alias downtest="(cd $testdir; docker-compose down) &"alias buildtest="cd $testdir; docker-compose build) &"
If you can not find the file  ̃/bashrc.txt create it with touch ̃/bashrc.txt.(MacOs or one of various Linux, or Unix operating systems.)
Note: Remember to source  ̃/.bashrc_profile and/or  ̃/bashrc.txt when you done editing them.
The updev command resulted in the following stream of messages on my terminal console. (Yours will have different timestamps and other slight differences.)
dev_1  | [I 15:53:33.389 NotebookApp] [jupyter_nbextensions_configurator] enabled 0.4.1dev_1  | [I 15:53:33.817 NotebookApp] JupyterLab extension loaded from /opt/conda/lib/python3.7/site-packages/jupyterlabdev_1  | [I 15:53:33.818 NotebookApp] JupyterLab application directory is /opt/conda/share/jupyter/labdev_1  | [I 15:53:34.196 NotebookApp] Serving notebooks from local directory: /dockerdev_1  | [I 15:53:34.196 NotebookApp] The Jupyter Notebook is running at:dev_1  | [I 15:53:34.196 NotebookApp] http://dd974d01052d:8888/?token=78824facb945e1bf386a6ead41f7b147d5ac4240dc673421dev_1  | [I 15:53:34.196 NotebookApp]  or http://127.0.0.1:8888/?token=78824facb945e1bf386a6ead41f7b147d5ac4240dc673421
Note: The updev command outputs:
..http://127.0.0.1:8888/?token=12377aa1c91b45e66e45eb5d5bdf2115cad2dae5b61be5c0
Because we map port 8888 to port 8889(Dev ) and port 8890(Test ) in the dockerfile.yaml. Users of DET 1.3.0 release must cut&paste the following into their web browser.
http://127.0.0.1:8889/?token=12377aa1c91b45e66e45eb5d5bdf2115cad2dae5b61be5c0
Does that seem a lot of detail to “Adding Jupyter Notebook Extensions to a Docker Image.” Perhaps it is. I recap:
Add the Nbextensionsyou want in the Jupyter user Dockerfile. TheNbextensions names and how to enable them are given above.Edit the dockerfile.yaml volume: - ./../../. ,if necessary. The subsequent launch of jupyter uses <./../../.as its top level directory.Optionally, “Automate the Frequently Used docker-compose commands” by adding them to your  ̃/.bashrc_profile file.Finally, enable the Nbextensions:
Add the Nbextensionsyou want in the Jupyter user Dockerfile. TheNbextensions names and how to enable them are given above.
Edit the dockerfile.yaml volume: - ./../../. ,if necessary. The subsequent launch of jupyter uses <./../../.as its top level directory.
Optionally, “Automate the Frequently Used docker-compose commands” by adding them to your  ̃/.bashrc_profile file.
Finally, enable the Nbextensions:
It took a couple of days of twist, turns, and wrong paths to get to DET 1.3.0 release. I hope you take less time to implement your Docker solution and, this article helps in your effort.
If you have a suggestion or improvements, please comment.
There is more detail on the Docker implementation I use in:
medium.com
medium.com
Note: You can adapt the Docker code to your project from a clone-able GitHub repo.