GSoC 2023 @ PyBaMM, NumFOCUS: Weeks 5 and 6
Tags: gsoc, python, opensource
Published: 2023-07-13 22:12:58

Hi there! This is the fourth blog in a series of seven blogs that document my journey of my Google Summer of Code project under the NumFOCUS umbrella organisation where I aim to improve the documentation infrastructure for PyBaMM, a Python package for working with physics-based battery models.

---

## A summary of the tasks I completed

I am continuing to work on adding quality-of-life improvements to the documentation and also to refine it, i.e., to consolidate all its parts in one place so that it is more readable and maintainable for core users and developers respectively.

1. Use `nbsphinx` to embed Jupyter notebooks ([pybamm-team/PyBaMM #3043](https://github.com/pybamm-team/PyBaMM/pull/3043)) was now merged. 
2. Display when pages were last updated in the docs ([pybamm-team/PyBaMM #3089](https://github.com/pybamm-team/PyBaMM/pull/3089))
3. Enable multithreaded documentation builds ([pybamm-team/PyBaMM #3097](https://github.com/pybamm-team/PyBaMM/pull/3097))
4. Add an option to download Jupyter notebooks from readthedocs ([pybamm-team/PyBaMM #3076](https://github.com/pybamm-team/PyBaMM/pull/3076))
5. Add floating window tooltips for cross-references in the docs ([pybamm-team/PyBaMM #3083](https://github.com/pybamm-team/PyBaMM/pull/3083))
6. Generate inheritance diagrams for models and submodels in the documentation ([pybamm-team/PyBaMM #3074](https://github.com/pybamm-team/PyBaMM/pull/3074))
7. Cache `nox` environments and refactor unit tests in CI ([#3079](https://github.com/pybamm-team/PyBaMM/pull/3079))

> PyBaMM also recently migrated from the DSL-based `tox` to the Python-based (infrastrucure-as-code) `nox` for running the test suite, thanks to my brilliant GSoC workmate Arjun. Many improvements to the testing infrastructure are planned, and I took on the task for one of them (more to come in the future!). I like this article titled [**Why I Like Nox**](https://hynek.me/articles/why-i-like-nox/) by the PSF Fellow Hynek Schlawack, that somewhat echoes the sentiment of why we decided to switch to it.

Some PRs in the [pybamm.org](https://pybamm.org) website [repository](github.com/pybamm-team/pybamm.org) that I opened, which will be completed in the coming weeks:

1. Generate teams ([pybamm-team/pybamm.org #31](https://github.com/pybamm-team/pybamm.org/pull/31))
2. Accessibility improvements ([pybamm-team/pybamm.org #32](https://github.com/pybamm-team/pybamm.org/pull/32))

The website is at the verge of its completion and will be showcased to the battery community and ecosystem soon!

---

## Gorgeous galleries for notebooks

In an attempt to incorporate the notebooks in the documentation, I had previously started adding the notebooks to the documentation. All the notebooks have been ported now and can be viewed on the [Example Notebooks page](https://docs.pybamm.org/en/latest/source/examples/notebooks/index.html) in the PyBaMM documentation.

I also configured the [`sphinx-gallery`](https://sphinx-gallery.github.io/stable/index.html) extension to generate thumbnails for each notebook. See it in action below:

![A white and black mixture of how the Jupyter notebook thumbnails look, this shows the Plotting, Solvers, and Spatial Methods notebooks](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/qzub3t5kfjwibs6lc3qb.png)

All the notebooks can be accessed through Google Colab as before. The example scripts will be ported in the coming weeks in order to finalise the thumbnails.

## Download the notebooks at ease

The `nbsphinx` extension also includes support for prologues and epilogues, i.e., text that can be embedded at the Sphinx builder runtime at either the top or the bottom of a particular page—in this case, the notebooks pages. I added an `nbsphinx_prolog` configuration value in `docs/source/conf.py` (the Sphinx configuration file) in this manner:

```python
nbsphinx_prolog = r"""

{% set readthedocs_download_url =
'https://docs.pybamm.org/en/latest/' %}

{% set doc_path = env.doc2path(env.docname, base=None) %}

.. raw:: html

    <div class="admonition tip">
        <p class="admonition-title">
            Tip
        </p>
        <p>
            You may
            <a href="{{ readthedocs_download_url | e }}{{ doc_path | e }}"
            target="_blank" download>
            download this notebook</a> and run it offline.
        </p>
    </div>

"""
```

The combination of raw HTML and Jinja templating, indubitably, allows us to do many useful things with Sphinx. Users can use the aforementioned _Open in Colab_ button as well to run the notebooks in the browser.

Here is an image of the admonition note in action:

![An admonition note at the top of the Jelly roll model Jupyter notebook page, which contains a Google Colab badge and a download link](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/wclwim0auxpb2g9xtxkw.png)

It can be tried on any of the notebooks on the [Example Notebooks page](https://docs.pybamm.org/en/latest/source/examples/notebooks/index.html) in the documentation.

## Version control history for the documentation pages

_...so that one can know when they messed things up, or when an archaic and unkempt API description was last taken care of. Essentially the `git blame` for documentation hosted on Read the Docs._

I added the `sphinx-last-updated-by-git` extension and configured it to be in the end of the footer for each page using the `html_theme_options` configuration value. The doctests did create some errors because GitHub Actions does not clone the PyBaMM repository to its full depth; it only pulls the latest 50 commits in its checkout step. A similar issue occurred on Read the Docs as well, which was ultimately fixed by adding a custom build command for the `post-checkout` step. This way, a complete rendition of the `Git` version control  history is downloaded while hosting.

## Speeding up the documentation builds

Now that the [`nbsphinx`](https://github.com/spatialaudio/nbsphinx) PR was merged, the `sphinx-build` commands for the documentation were between 1.5x–2x slower, which subsequently became a menace when building the documentation locally and while running the doctests. The reason behind this was that `nbsphinx` was re-building all the notebooks pages and uses [`Pandoc`](https://pandoc.org) under the hood to convert cell-based Markdown to `.rst` formats. We had decided not to use the other notebooks rendering solution, i.e., a rival project called [`MyST-NB`](https://github.com/executablebooks/MyST-NB), because it was not compatible with `sphinx-gallery`. Its advantage would have been that it has native caching by default and would not slow down the documentation builds by a significant amount. 

In a short PR, eventually enabled multithreading for building the documentation using the `-j auto` flag in SPHINXOPTS, and the results bestowed, to my delight, much faster build times (~40%)! The time came down from the previous ~3.5 minutes to ~2 minutes. Please see the mentioned issue above for more details.

## Traversing the maze of PyBaMM's many, many battery models with inheritance diagrams

PyBaMM comes with an extensive library of models and submodels, such as the Doyle-Fuller-Newmann model, the Single Particle Model, lead-acid models, submodels for oxygen diffusion, particle cracking models, and many more. It is often difficult for inexperienced users who might still be learning the ropes of how to use PyBaMM effectively. To tackle this issue, I wrote a simple Sphinx extension and added it to the `docs/sphinxext` folder. It relies on appending and/or inserting particular lines of code through the `sphinx.ext.autodoc` extension, which provides functionality to include documentation for classes and methods by extracting their docstrings. 

I used a dropdown from the `sphinx-design` extension to make the inheritance diagram collapsible upon request and through the click of a button. The extension uses `graphviz` and `sphinx.ext.inheritance_diagram` as dependencies to create the diagram and can be configured in `conf.py`. Currently, the extension creates these plots in `.PNG` format for the models and the submodels.

Here is an example for the Many Particle Model (MPM):

![API documentation for the Many Particle Model, which shows its inheritance diagram, expanded](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/w3g87qiyg815l1ehwahp.png)

## Floating-window tooltips for hyperlinks in the documentation

It is also necessary to sometimes have a quick look at a particular parameter for a method or a class, but one does not want to search for it and go to the results. To enhance the productivity for PyBaMM users and developers, I added and configured a Sphinx extension `sphinx-hoverxref`, which adds tooltips that can be viewed upon hovering on a particular link within the _same_ tab. See it in action below in the Interpolant class in PyBaMM, which is used heavily in the expression tree and its operators:

![Floating-window tooltip is visible for the Symbol class, on the same page as the Interpolant class in the Expression Tree API documentation](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/saqyqfhg2hn6h2cp7umv.png)

## Tasks to focus on in the coming weeks

1. Implement a `search-as-you-type` feature for the hosted documentation for relatively faster search results in comparison to the default JavaScript-based search engine
2. Make the documentation more maintainable using `sphinx.ext.autosummary`

## Stay tuned

I will be posting these blogs fortnightly as a part of my GSoC project. Thoughts, comments, constructive criticism, and insights will be greatly appreciated. See you soon on the next one! For updates, please follow me on my social media handles:

- [Twitter](https://twitter.com/agriyakhetarpal)
- [GitHub](https://github.com/agriyakhetarpal)
- [LinkedIn](https://linkedin.com/in/agriyakhetarpal)