Package: nbwipers
nbwipersnbwipers is a command line tool to wipe clean jupyter notebooks, written in Rust.The interface and functionality are based onnbstripoutand the idea to implement it in rust comes fromnbstripout-fast.Usagenbwipers has a few subcommands that provide functionality related to cleaning Jupyter notebooks.clean: clean a single notebook. This is more-or-less equivalent tonbstripoutcheck: check notebooks in a given path for elements that would be removed byclean. This could be used in a CI context to enforce clean notebooks.clean-allclean all notebooks in a given path. This one should be used carefully!installregister nbwipers as a git filter foripynbfiles. Equivalent tonbstripout --installuninstallremove nbwipers as a git filtercheck-installcheck thatnbwipersornbstripoutis installed in the local repo. This is used in the pre-commit hook.The full options can be found inCommandLineHelp.md.ExamplesTo set up nbwipers as a git filter in your repository, usenbwipersinstalllocalTo check the notebooks in your foldernbwiperscheck.pre-commitYou can add the following to yourpre-commit-config.yamlfile to ensure thatnbwipersornbstripoutis installed in your repo, in order to prevent Jupyter notebook outputs from being committed to version control.-repo:https://github.com/felixgwilliams/nbwipers-pre-commitrev:v0.3.3hooks:-id:nbwipers-check-installAlternatively, you can use the URL for this repo in your config, but this will compilenbwipersfrom source, rather than retrieving the binary from PyPI, and is therefore not recommended.If you are using your pre-commit configuration as part of CI, you should set the environment variableNBWIPERS_CHECK_INSTALL_EXIT_ZEROwhich forces this check to pass, since you do not neednbwipersconfigured in your CI environment.MotivationA working copy of a Jupyter notebook containsCode written by the authorNotebook outputs: tables, logs, tracebacks, images, widgets and so on...Execution countsMetadata, such as whether cells are collapsed, scrollable etc.Of these categories of data, only the first — code written by the author — should definitely be tracked by version control, since it is the product of the author's intension and hard work.
The other categories of data are subject to change outside of the explicit intensions of the author, and are generally noisy from a version control perspective.Moreover, including notebook outputs in version controlmakes diffs harder to interpret, as they will contain lots of unintended changesincreases the risk of a tricky merge conflict if different users run the same cell and get a slightly different resultincreases the amount of data committed, which can degrade repository performancerisks leaking sensitive dataBy using nbwipers or nbstripout as a git filter, the problematic parts of the notebook are removed from the version of the file that git sees, while leaving your working copy intact.Testing CoverageTo test coverage, use the command:cargotarpaulin-ostdout-ohtml-olcov--enginellvmUsing thellvmengine means that integration tests contribute to coverage.Acknowledgementsnbwipers relies on inspiration and code from several projects.
For the projects whose code was used please seeLICENSEfor the third-party notices.nbstripoutstrip output from Jupyter and IPython notebooksnbstripout is an invaluable tool for working with Jupyter Notebooks in the context of version control.
This project forms the basis of the interface and logic of this project and is also the source of the testing examples.nbstripout-fastA much faster version of nbstripout by writing it in rust (of course).nbstripout-fast, like this project, implements the functionality of nbstripout in Rust, while also allowing repo-level configuration in a YAML file.With nbwipers I hoped to recreate the idea of nbstripout-fast, but with the ability to install as a git filter, and configuration viapyproject.toml.ruffAn extremely fast Python linter and code formatter, written in Rust.Ruff is quickly becomingthelinter for python code, thanks to its performance, extensive set of rules and its ease of use.
It was a definite source of knowledge for the organisation of the configuration and the file discovery.
The schema for Jupyter Notebooks, and some of the file discovery code was adapted from Ruff.