Tips, Tricks, Hacks, and Magic: How to Effortlessly Optimize Your Jupyter Notebook | by Anne Bonner | Towards Data Science

The really cool thing about tech is how many people are out there working hard to make your life more fun. Every minute of every day, there are people putting their blood, sweat, and tears into tools that will make your programs, packages, apps, and life run more smoothly.
You might, for example, think that once you have Jupyter Notebooks up and running, that’s it. If they work, you’re done! What you might not realize is that there are almost endless ways that you can customize your notebooks. Getting a program or package installed is just the beginning!
Why not take a few minutes to get comfortable and make a few improvements? There are a ton of simple ways to quickly make your Jupyter Notebooks better, faster, stronger, sexier, and so much more fun to work with.
This guide assumes that you’re pretty new to Jupyter Notebooks. We’ll start with the really basic stuff for beginners before we move into the cooler tricks. If you’re really, really new and having trouble getting Anaconda installed and working, you might want to check out this article:
towardsdatascience.com
After you get everything installed, any time you want to launch Jupyter Notebook, you can just open up your terminal and run
jupyter notebook
and you’ll be up and running!
One of the first things people want to change in their Jupyter notebooks is the theme. People are crazy about dark mode! That’s incredibly easy and you can switch it up any time you want to.
First, go to your terminal and install Jupyterthemes with
pip install jupyterthemes
Now you can install the super popular dark theme with
jt -t chesterish
Restore the main theme any time with
jt -r
Click here to find the Jupyterthemes GitHub repo.
You can quickly access keyboard shortcuts with the command palette. just type Ctrl + Shift + P or Cmd + Shift + P to access a dialog box that’s a lot like Spotlight Search on a Mac. It can help you run any command by name, which is great if you don’t know the keyboard shortcut.
Shift + Enter lets you run the current cell
Esc takes you into command mode. Now you can navigate around your notebook with your arrow keys!
In command mode, use
A to insert a new cell above your current cell
B to insert a new cell below your current cell
M to change the current cell to Markdown
Y to change back to a code cell
D + D to delete the current cell (press the key twice)
Enter takes you from command mode back into edit mode
Also
Shift + Tab will show you the documentation for the object you just typed into a code cell. (You can keep pressing this to cycle through a few modes.)
Esc + F helps you find and replace info in your code (not in the outputs)
Esc + 0 Toggles cell output
Shift + J or Shift + Down selects the next cell in a downwards direction. Shift + K or Shift + Up selects cells in an upwards direction. Once your cells are selected, you can delete/ copy/cut/paste/run them as a batch. That’s awesome when you need to move parts of a notebook!
Shift + M lets you merge multiple cells. (If you just try to click on the cells you want to work you’ll have trouble. Hold down the shift key and click the cells you want to merge. Then, while you’re still holding down the shift key, press M.)
Also, you can run bash commands in a notebook if you put an exclamation point at the beginning. For example: !pip install numpy
You can suppress the output of the function on a final line of code any time by adding a semicolon at the end.
You might want to add new lines of code and comment out the old lines while you’re working. This is great if you’re improving the performance of your code or trying to debug it.
First, select all the lines you want to comment out.
Next hit cmd + / to comment out the highlighted code!
You can write LaTex in a Markdown cell any time and it will be rendered as a formula.
That changes this
$P(A \mid B) = \frac{P(B \mid A)P(A)}{P(B)}$
into this
Normally only the last output in the cell will be printed. For everything else, you have to manually add print(), which is fine but not super convenient. You can change that by adding this at the top of the notebook:
from IPython.core.interactiveshell import InteractiveShellInteractiveShell.ast_node_interactivity = "all"
This means that, while normally you’d only get one output printed
Now you’ll see both outputs!
Any time you want to go back to the original setting, just run
from IPython.core.interactiveshell import InteractiveShellInteractiveShell.ast_node_interactivity = "last_expr"
Just be aware that you have to run the setting change in a separate cell for it to take effect for the next cell run.
Because it’s an open source web app, a ton of extensions have been developed for Jupyter Notebooks. You can find the official iPython extension list here. This is another popular bundle of extensions.
You can install Nbextensions any time from your command line like this
with pip
pip install jupyter_contrib_nbextensionsjupyter contrib nbextension install --user
or with Anaconda
conda install -c conda-forge jupyter_contrib_nbextensionsconda install -c conda-forge jupyter_nbextensions_configuratorjupyter contrib nbextension install --user
Once they’re installed, you’ll see an Nbextensions tab. Explore away!
Head on over here to read more about the extensions and how to enable them, disable them, and more.
I won’t go into too much detail about adding and enabling extensions and how to use them because it’s so incredibly well explained right in your Jupyter Notebook! Just click on “Nbextensions” at the top of the screen, click on the extension you’re interested in, and then scroll down for the information you need and a GIF of the extension in action!
Scratchpad — This is awesome. It allows you to create a temporary cell to do quick calculations without creating a new cell in your workbook. This is a huge time saver!
Hinterland — This enables a code autocompletion menu for every keypress in a code cell instead of just with tab
Snippets — Adds a drop-down menu to insert snippet cells into the current notebook.
Autopep8 — This is a tool that automatically formats Python code to conform to the PEP 8 style guide. It’s so handy! Make sure you have run pip install autopep8 --user on your local machine. This will make sure you’re following the correct python coding conventions.
Split Cells Notebook — Enables split cells in Jupyter notebooks. Enter command mode and use Shift + S to toggle the current cell to either a split cell or full width.
Table of Contents — This extension enables you to collect all running headers and display them in a floating window, as a sidebar, or with a navigation menu.
A Code Prettifier — Cleans up, formats, and indents your code, so you don’t have to.
Notify — This displays a desktop notification when your kernel becomes idle. This is awesome when you’re running code that takes more than a couple of seconds to complete.
Code Folding — While in edit mode, a triangle appears in the gutter to fold your code. Good when you have large functions you want to hide for readability.
Zen mode — Makes things a bit less cluttered. Make sure to turn off the backgrounds in the settings.
Magics are handy commands that make life easier when you want to perform particular tasks. They often look like unix commands, but they’re all implemented in Python. There are a ton of magics out there!
There are two kinds of magic: line magic (use this on one line) and cell magic (this applies to the whole cell). Line magics start with a percent character %, and cell magics start with two, %%. To see the available magics, run:
%lsmagic
You can easily manage the environment variables of your notebook without restarting anything with %env. If you run it without any variables, it will list all of your environment variables.
You can insert code from an external script with %load. (More on this below, but it’s awesome, so I’m adding it up here) For example:
%load basic_imports.py
will grab the basic_imports.py file and load it in your notebook!
This is unbelievably helpful. With almost no effort, you can export the contents of a cell any time with %%writefile. For example
%%writefile thiscode.pyyou'd write some cool code or function in here    that you'd want to export    and maybe even use later!
Do you find yourself running the same imports in every notebook or adding the same function all the time? Now you can write it once and use it everywhere!
You can write a file basic_imports.py containing the following code:
%writefile basic_imports.pyimport pandas as pdimport numpy as npimport matplotlib.pyplot as plt
That will create a .py file that contains your basic imports.
You can load this any time by writing:
%load basic_imports.py
Executing this replaces the cell contents with the loaded file.
# %load imports.pyimport pandas as pdimport numpy as npimport matplotlib.pyplot as plt
Now we can run the cell again to import all our modules and we’re ready to go.
Like most people, you probably find yourself writing the same few tasks over and over again. Maybe there are a few equations that you find yourself computing repeatedly or some lines of code that you’ve produced countless times. Jupyter lets you save code snippets as executable macros! Macros are just code, so they can contain variables that will have to be defined before execution. Let’s define one!
Let’s say
name = 'Kitten'
Now, to define a macro we need some code to use. We can save pretty much anything, from a string to a function, or whatever you need.
print('Hello, %s!' % name)Hello, Kitten!
We use the %macro and %load magics to set up a reusable macro. It’s pretty common to begin macro names with a double underscore to distinguish them from other variables.
%macro -q __hello_you 32
The %macro magic takes a name and a cell number (or numbers), and we also passed -q to make it less verbose. %store allows us to save any variable for use in other sessions. Here we passed the name of the macro we created so we can use it again after the kernel is shut down or in other notebooks.
To load the macro, we just run:
%load __hello_you
And to execute it, we can just run a cell that contains the macro name.
__hello_youHello, Kitten!
Success!
Let’s modify the variable we used in the macro.
name = 'Muffins'
When we run the macro now, our modified value is picked up.
__hello_youHello, Muffins!
This works because macros execute the saved code in the scope of the cell. If name was undefined we’d get an error.
Want to use that same macro across all of your notebooks?
%store lets you store your macro and use it across all of your Jupyter Notebooks.
Now you can open a new notebook and try it out with %store -r __hello_you. Load that baby up and you’re ready to go!
%store -r __hello_youname = 'Rambo'%load __hello_youHello, Rambo!
%run magic will execute your code and display any output, including Matplotlib plots. You could even execute entire notebooks this way.
%run can execute python code from .py files. It can also execute other Jupyter notebooks.
You can use %pycat any time to show the contents of a script if you aren’t sure what’s in there.
%pycat basic_imports.py
The %autosave magic lets you change how often your notebook will auto save to its checkpoint file.
%autosave 60
That will set you to autosave every 60 seconds.
%matplotlib inline
You probably already know this one, but %matplotlib inline will display your Matplotlib plot images right in your cell outputs. That means that you can include Matplotlib charts and graphs right in your notebooks. It makes sense to run this at the beginning of your notebook, right in the first cell.
There are two IPython Magic commands that are useful for timing — %%time and %timeit. These are seriously useful when you have some slow code and you’re trying to identify where the issue is. They both have line and cell modes.
The main difference between %timeit and %time is that %timeit runs the specified code many times and computes an average.
%%time will give you information about a single run of the code in your cell.
%%timeit uses the Python timeit module which runs a statement a ton of times and then provides the mean of the results. You can specify the number of runs with the -n option, -r to specify the number of repeats, and more.
You can also execute a cell using the specified language. There are extensions available for several languages. You have options like
%%bash
%%HTML
%%python
%%python2
%%python3
%%ruby
%%perl
%%capture
%%javascript
%%js
%%latex
%%markdown
%%pypy
To render HTML in your notebook, for example, you’d run:
%%HTMLThis is <em>really</em> neat!
You can also use LaTeX directly any time you want to with
%%latexThis is an equation: $E = mc^2$
The %who command without any arguments will list all variables that exist in the global scope. Passing a parameter like str will list only variables of that type. So if you type something like
%who str
in our notebook, you’d see
%prun shows how much time your program spent in each function. Using %prun statement_name gives you an ordered table showing the number of times each internal function was called within the statement, the time each call took, and the cumulative time of all runs of the function.
Jupyter has own interface for The Python Debugger. This makes it possible to go inside the function and take a look at what happens there. You can turn that on by running %pdb at the top of the cell.
One simple line of IPython magic will give you double resolution plot output for Retina screens. Just be aware that this won’t render on non-retina screens.
%config InlineBackend.figure_format ='retina'
Just add %%script false at the top of the cell
%%script falseyou'd put some long     code here that you don't want to run     right now
This is actually a Python trick, but you might want it for when you’re running code that’s taking forever. If you don’t want to be staring at your code all day, but you need to know when it’s done, you can make your code sound an “alarm” when it’s finished!
On Linux (and Mac)
import osduration = 1  # secondfreq = 440  # Hzos.system('play --no-show-progress --null --channels 1 synth %s sine %f' % (duration, freq))
On Windows
import winsoundduration = 1000  # millisecondfreq = 440  # Hzwinsound.Beep(freq, duration)
In order to use this, you need to install sox, which you should be able to do with
brew install sox
...assuming you have homebrew installed. If you haven’t taken any time to customize and improve your terminal, you might want to check out this article!
towardsdatascience.com
That should be enough to get you started! If you know any tips and tricks that you think could help other beginners, let everyone know about them in the comments below. Your options here are endless!
If you really want to take things to the next level, you might want to check out this post:
towardsdatascience.com
And if you’re interested in building interactive dashboards, check out this one:
blog.dominodatalab.com
Thanks for reading! If you want to reach out or find more cool articles, please come and join me at Content Simplicity!