nbdev formating and linting
Published: 2023-01-06 21:04:35

-   <a href="#introduction" id="toc-introduction">Introduction</a>
-   <a href="#nbqa" id="toc-nbqa">nbqa</a>
-   <a href="#black" id="toc-black">black</a>
-   <a href="#isort-and-mypy" id="toc-isort-and-mypy">isort and mypy</a>
-   <a href="#automating-your-flows-with-makefile"
    id="toc-automating-your-flows-with-makefile">Automating your flows with
    Makefile</a>
-   <a href="#pre-commit" id="toc-pre-commit">pre-commit</a>
    -   <a href="#install-and-configure-pre-commit"
        id="toc-install-and-configure-pre-commit">install and configure
        pre-commit</a>
    -   <a href="#create-.pre-commit-config.yaml"
        id="toc-create-.pre-commit-config.yaml">create
        .pre-commit-config.yaml</a>
-   <a href="#reproducible-environments-with-pipenvpoetryvirtualenv"
    id="toc-reproducible-environments-with-pipenvpoetryvirtualenv">reproducible
    environments with Pipenv/Poetry/virtualenv</a>
-   <a href="#concusion" id="toc-concusion">Concusion</a>

## Introduction

nbdev is a great tool.

However some of you may not be comfortable with the default [fastai
codestyle](https://docs.fast.ai/dev/style.html), or maybe your team is
already using a different one.

This post will show you how to integrate your nbdev workflows with
industry standard tools like:

-   [black](https://github.com/psf/black) , The uncompromising code
    formatter
-   [isort](https://pycqa.github.io/isort/) , A Python utility / library
    to sort imports.
-   [mypy](http://www.mypy-lang.org/) , Optional static typing for
    Python
-   [pre-commit](https://pre-commit.com/) , A framework for managing and
    maintaining multi-language pre-commit hooks.

> **Note**
>
> An astute reader may notice that one can enable
> `black_formatting = True` in `settings.ini`
>
> However,
>
> -   this will only format the code in the module, not the code in the
>     notebook.
>
> -   this does not allow you to use other tools like `isort` or `mypy`

## nbqa

use [nbqa](https://nbqa.readthedocs.io/en/latest/) to format and verify
code in your notebooks

nbqa is a tool that allows you to run any tool (formatter/linter) on
your notebooks, without having to worry about the details.

## black

Black is a python code formatter that is becoming the de-facto standard

``` bash
pip install nbqa

nbqa black nbs/
```

## isort and mypy

isort will sort your imports, and mypy will check your type annotations

``` bash
pip install isort mypy

nbqa isort nbs/
nbqa mypy nbs/
```

## Automating your flows with Makefile

But you are already using nbdev command-line arguments like
`nbdev_export` and `nbdev_test`, who wants to remember more commands?

[Make](https://www.gnu.org/software/make/) is a great tool for
automating your workflows, usually comes pre-installed on most unix
systems.

Create a `Makefile` in the root of your project and add the following
lines:

**Makefile**

``` makefile
# Makefile
format:
    nbqa black nbs/
    nbqa isort nbs/
    nbqa mypy nbs/
```

Now you can run `make format` to format your notebooks.

Let’s add more commands to our `Makefile`:

**Makefile**

``` makefile
format:
    nbqa black nbs/
    nbqa isort nbs/
    nbqa mypy nbs/

export:
    nbdev_export

mypy:
    nbqa mypy nbs/ --ignore-missing-imports --check-untyped-defs

test:
    nbdev_test --n_workers 4

prepare:
    # Export, test, and clean notebooks, and render README if needed
    nbdev_prepare

docs:
    nbdev_docs
```

And now the cherry on top: Makefile supports dependecies, so we can
chain commands together.

add this to your `Makefile`:

**Makefile**

``` makefile
all: format prepare mypy
```

Running `make all` will format your notebooks, export them, run mypy,
run tests and update README.md

## pre-commit

Want to automate your formatting even more?

Force your team to format their notebooks before committing?

Use [pre-commit](https://pre-commit.com/)

### install and configure pre-commit

``` bash
pip install pre-commit # install 

pre-commit install # enable for this repo
```

### create .pre-commit-config.yaml

**.pre-commit-config.yaml**

``` yaml
repos:
  - repo: https://github.com/fastai/nbdev
    rev: 2.2.10
    hooks:
      - id: nbdev_clean
      - id: nbdev_export
```

**Add nbqa hooks**

**.pre-commit-config.yaml**

``` yaml
  - repo: https://github.com/nbQA-dev/nbQA
    rev: 1.5.3
    hooks:
      - id: nbqa-mypy
        args: ["--ignore-missing-imports", "--check-untyped-defs"]
        additional_dependencies: ["mypy", "types-toml", "types-requests"]
      - id: nbqa-black
      - id: nbqa-isort
```

**control your hooks behaviour**

**.pre-commit-config.yaml**

``` yaml
  - repo: https://github.com/nbQA-dev/nbQA
    rev: 1.5.3
    hooks:
      - id: nbqa-mypy
        args: ["--ignore-missing-imports", "--check-untyped-defs"] # pass args to mypy
        exclude: "directory_to_exclude/.*" # eclude any directory/files from formatting
        additional_dependencies: ["mypy", "types-toml", "types-requests"] # install additional dependencies
      - id: nbqa-black
        exclude: "directory_to_exclude/.*" # eclude any directory/files from formatting
      - id: nbqa-isort
        exclude: "directory_to_exclude/.*" # eclude any directory/files from formatting
```

## reproducible environments with Pipenv/Poetry/virtualenv

Sometimes you may run into reproducibility issues, where your notebook
works on your machine, but not on your team’s.

The best way to avoid it is to use a dependency manager, such as:

-   [Pipenv](https://pipenv.pypa.io/en/latest/index.html)
-   [Poetry](https://python-poetry.org/)
-   [virtualenv](https://virtualenv.pypa.io/en/latest/)

But how do you integrate it with nbdev, since it uses `settings.ini` to
manage dependencies?

In [this blog post](https://dev.to/ddobrinskiy/reproducible-environment-in-nbdev-with-pipenv-4h05) I show an example of
how to use Pipenv with nbdev.

## Concusion

I hope you find this post useful.

These changes may make nbdev more friendly to a team environment.

Please don’t hesitate to post any questions or comments below.