Bringing the best out of Jupyter Notebooks for Data Science | by Parul Pandey | Towards Data Science

Reimagining what a Jupyter notebook can be and what can be done with it.
Netflix aims to provide personalized content to their 130 million viewers. One of the significant ways by which data scientists and engineers at Netflix interact with their data is through Jupyter notebooks. Notebooks leverage the use of collaborative, extensible, scalable, and reproducible data science. For many of us, Jupyter Notebooks is the de facto platform when it comes to quick prototyping and exploratory analysis. However, there’s more to this than meets the eye. A lot of Jupyter functionalities sometimes lies under the hood and is not adequately explored. Let us try and explore Jupyter Notebooks’ features which can enhance our productivity while working with them.
Executing Shell CommandsJupyter ThemesNotebook ExtensionsJupyter WidgetsQgridSlideshowEmbedding URLs, PDFs, and Youtube Videos
Executing Shell Commands
Jupyter Themes
Notebook Extensions
Jupyter Widgets
Qgrid
Slideshow
Embedding URLs, PDFs, and Youtube Videos
The notebook is the new shell
The shell is a way to interact textually with the computer. The most popular Unix shell is Bash(Bourne Again SHell ). Bash is the default shell on most modern implementations of Unix and in most packages that provide Unix-like tools for Windows.
Now, when we work with any Python interpreter, we need to regularly switch between the shell and the IDLE, in case we need to use the command line tools. However, the Jupyter Notebook gives us the ease to execute shell commands from within the notebook by placing an extra !before the commands. Any command that works at the command-line can be used in IPython by prefixing it with the ! character.
In [1]: !lsexample.jpeg list tmpIn [2]: !pwd/home/Parul/Desktop/Hello World Folder'In [3]: !echo "Hello World"Hello World
We can even pass values to and from the shell as follows:
In [4]: files= !lsIn [5]: print(files)['example.jpeg', 'list', 'tmp']In [6]: directory = !pwdIn [7]: print(directory)['/Users/Parul/Desktop/Hello World Folder']In [8]: type(directory)IPython.utils.text.SList
Notice, the data type of the returned results is not a list.
Theme-ify your Jupyter Notebooks!
If you are a person who gets bored while staring at the white background of the Jupyter notebook, themes are just for you. The themes also enhance the presentation of the code. You can find more about Jupyter themes here. Let’s get to the working part.
Installation
pip install jupyterthemes
List of available themes
jt -l
Currently, the available themes are chesterish, grade3, gruvboxd, gruvboxl monokai, oceans16, onedork, solarizedd ,solarizedl.
# selecting a particular themejt -t <name of the theme># reverting to original Themejt -r
You will have to reload the jupyter notebook everytime you change the theme, to see the effect take place.
The same commands can also be run from within the Jupyter Notebook by placing ‘!’ before the command.
Extend the possibilities
Notebook extensions let you move beyond the general vanilla way of using the Jupyter Notebooks. Notebook extensions (or nbextensions) are JavaScript modules that you can load on most of the views in your Notebook’s frontend. These extensions modify the user experience and interface.
Installation with conda:
conda install -c conda-forge jupyter_nbextensions_configurator
Or with pip:
pip install jupyter_contrib_nbextensions && jupyter contrib nbextension install#incase you get permission errors on MacOS,pip install jupyter_contrib_nbextensions && jupyter contrib nbextension install --user
Start a Jupyter notebook now, and you should be able to see an NBextensions Tab with a lot of options. Click the ones you want and see the magic happen.
In case you couldn’t find the tab, a second small nbextension, can be located under the menuEdit.
Let us discuss some of the useful extensions.
Hinterland enables code autocompletion menu for every keypress in a code cell, instead of only calling it with the tab. This makes Jupyter notebook’s autocompletion behave like other popular IDEs such as PyCharm.
This extension adds a drop-down menu to the Notebook toolbar that allows easy insertion of code snippet cells into the current notebook.
This extension splits the cells of the notebook and places then adjacent to each other.
This extension enables to collect all running headers and display them in a floating window, as a sidebar or with a navigation menu. The extension is also draggable, resizable, collapsible and dockable.
Collapsible Headings allows the notebook to have collapsible sections, separated by headings. So in case you have a lot of dirty code in your notebook, you can simply collapse it to avoid scrolling it again and again.
Autopep8 helps to reformat/prettify the contents of code cells with just a click. If you are tired of hitting the spacebar again and again to format the code, autopep8 is your savior.
Make notebooks interactive
Widgets are eventful python objects that have a representation in the browser, often as a control like a slider, textbox, etc. Widgets can be used to build interactive GUIs for the notebooks.
# pippip install ipywidgetsjupyter nbextension enable --py widgetsnbextension# Condaconda install -c conda-forge ipywidgets#Installing ipywidgets with conda automatically enables the extension
Let us have a look at some of the widgets. For complete details, you can visit their Github repository.
Interact
The interact function (ipywidgets.interact) automatically creates a user interface (UI) controls for exploring code and data interactively. It is the easiest way to get started using IPython's widgets.
# Start with some imports!from ipywidgets import interactimport ipywidgets as widgets
def f(x):    return x# Generate a slider interact(f, x=10,);
# Booleans generate check-boxesinteract(f, x=True);
# Strings generate text areasinteract(f, x='Hi there!');
Here is a list of some of the useful advanced widgets.
The Play widget is useful to perform animations by iterating on a sequence of integers at a certain speed. The value of the slider below is linked to the player.
play = widgets.Play(    # interval=10,    value=50,    min=0,    max=100,    step=1,    description="Press play",    disabled=False)slider = widgets.IntSlider()widgets.jslink((play, 'value'), (slider, 'value'))widgets.HBox([play, slider])
The date picker widget works in Chrome and IE Edge but does not currently work in Firefox or Safari because they do not support the HTML date input field.
widgets.DatePicker(    description='Pick a Date',    disabled=False)
widgets.ColorPicker(    concise=False,    description='Pick a color',    value='blue',    disabled=False)
tab_contents = ['P0', 'P1', 'P2', 'P3', 'P4']children = [widgets.Text(description=name) for name in tab_contents]tab = widgets.Tab()tab.children = childrenfor i in range(len(children)):    tab.set_title(i, str(i))tab
Make Data frames intuitive
Qgrid is also a Jupyter notebook widget but mainly focussed at dataframes. It uses SlickGrid to render pandas DataFrames within a Jupyter notebook. This allows you to explore your DataFrames with intuitive scrolling, sorting and filtering controls, as well as edit your DataFrames by double-clicking cells. The Github Repository contains more details and examples.
Installing with pip:
pip install qgridjupyter nbextension enable --py --sys-prefix qgrid# only required if you have not enabled the ipywidgets nbextension yetjupyter nbextension enable --py --sys-prefix widgetsnbextension
Installing with conda:
# only required if you have not added conda-forge to your channels yetconda config --add channels conda-forgeconda install qgrid
Code is great when communicated.
Notebooks are an effective tool for teaching and writing explainable codes. However, when we want to present our work either we display our entire notebook(with all the codes) or we take the help of powerpoint. Not any more. Jupyter Notebooks can be easily converted to slides and we can easily choose what to show and what to hide from the notebooks.
There are two ways to convert the notebooks into slides:
Open a new notebook and navigate to View → Cell Toolbar → Slideshow. A light grey bar appears on top of each cell, and you can customize the slides.
Now go to the directory where the notebook is present and enter the following code:
jupyter nbconvert *.ipynb --to slides --post serve# insert your notebook name instead of *.ipynb
The slides get displayed at port 8000. Also, a .html file will be generated in the directory, and you can also access the slides from there.
This would look even more classy with a themed background. Let us apply the theme ’onedork’ to the notebook and then convert it into a slideshow.
These slides have a drawback i.e. you can see the code but cannot edit it. RISE plugin offers a solution.
RISE is an acronym for Reveal.js — Jupyter/IPython Slideshow Extension. It utilized the reveal.js to run the slideshow. This is super useful since it also gives the ability to run the code without having to exit the slideshow.
Installation
1 — Using conda (recommended):
conda install -c damianavila82 rise
2 — Using pip (less recommended):
pip install RISE
and then two more steps to install the JS and CSS in the proper places:
jupyter-nbextension install rise --py --sys-prefix#enable the nbextension:jupyter-nbextension enable rise --py --sys-prefix
Let us now use RISE for the interactive slideshow. We shall re-open the Jupyter Notebook we created earlier. Now we notice a new extension that says “Enter/Exit RISE Slideshow.”
Click on it, and you are good to go. Welcome to the world of interactive slides.
Refer to the documentation for more information.
Display it right there!
Why go with mere links when you can easily embed an URL, pdf, and videos into your Jupyter Notebooks using IPython’s display module.
#Note that http urls will not be displayed. Only https are allowed inside the Iframefrom IPython.display import IFrameIFrame('https://en.wikipedia.org/wiki/HTTPS', width=800, height=450)
from IPython.display import IFrameIFrame('https://arxiv.org/pdf/1406.2661.pdf', width=800, height=450)
from IPython.display import YouTubeVideoYouTubeVideo('mJeNghZXtMo', width=800, height=300)
These were some of the features of the Jupyter Notebooks that I found useful and worth sharing. Some of them would be obvious to you while some may be new. So, go ahead and experiment with them. Hopefully, they will be able to save you some time and give you a better UI experience. Also feel free to suggest other useful features in the comments.