nrtk 0.27.1
Natural Robustness Toolkit (NRTK) is a platform for generating validated, sensor-specific perturbations and transformations used to evaluate the robustness of computer vision models.
Requires Python: <3.14,>=3.10

![nrtk-logo](https://raw.githubusercontent.com/Kitware/nrtk/main/docs/figures/nrtk-wordmark.png)

<hr/>

<!-- :auto badges: -->

[![PyPI - Python Version](https://img.shields.io/pypi/v/nrtk)](https://pypi.org/project/nrtk/)
![PyPI - Python Version](https://img.shields.io/pypi/pyversions/nrtk)
[![Documentation Status](https://readthedocs.org/projects/nrtk/badge/?version=latest)](https://nrtk.readthedocs.io/en/latest/?badge=latest)

<!-- :auto badges: -->

# Natural Robustness Toolkit (NRTK)

> The Natural Robustness Toolkit (NRTK) is an open source toolkit for generating
> operationally realistic perturbations to evaluate the natural robustness of
> computer vision algorithms.

The `nrtk` package evaluates the natural robustness of computer vision
algorithms to various perturbations, including sensor-specific changes to camera
focal length, aperture diameter, etc.

We have also created `nrtk.interop` module to support AI T&E use cases and
workflows, through interoperability with
[MAITE](https://github.com/mit-ll-ai-technology/maite) and integration with
other [JATIC](https://cdao.pages.jatic.net/public/) tools. Users seeking to use
NRTK to perturb MAITE-wrapped datasets or evaluate MAITE-wrapped models should
utilize this module. Explore our
[T&E guides](https://nrtk.readthedocs.io/en/latest/tutorials/testing_and_evaluation_notebooks.html)
which demonstrate how `nrtk` perturbations and `maite` can be applied to assess
operational risks.

## Why NRTK?

NRTK addresses the critical gap in evaluating computer vision model resilience
to real-world operational conditions beyond what traditional image augmentation
libraries cover. T&E engineers need precise methods to assess how models respond
to sensor-specific variables (focal length, aperture diameter, pixel pitch) and
environmental factors without the prohibitive costs of exhaustive data
collection. NRTK leverages pyBSM's physics-based models to rigorously simulate
how imaging sensors capture and process light, enabling systematic robustness
testing across parameter sweeps, identification of performance boundaries, and
visualization of model degradation. This capability is particularly valuable for
satellite and aerial imaging applications, where engineers can simulate
hypothetical sensor configurations to support cost-performance trade-off
analysis during system design—ensuring AI models maintain reliability when
deployed on actual hardware facing natural perturbations in the field.

## Target Audience

This toolkit is intended to help data scientists, developers, and T&E engineers
who want to rigorously evaluate and enhance the robustness of their computer
vision models. For users of the JATIC product suite, this toolkit is used to
assess model robustness against natural perturbations.

<!-- :auto installation: -->

## Installation

`nrtk` installation has been tested on Unix and Linux systems.

To install the current version via `pip`:

```bash
pip install nrtk
```

To install the current version via `conda-forge`:

```bash
conda install -c conda-forge nrtk
```

This installs core functionality, but many specific perturbers require
additional dependencies.

### Installation with Optional Features (Extras)

NRTK uses optional "extras" to avoid installing unncessary dependencies. You can
install extras with square brackets:

```bash
# Install with extras (note: no spaces after commas)
pip install nrtk[<extra1>,<extra2>]
```

#### Common Installation Patterns

```bash
# For basic OpenCV image perturbations
pip install nrtk[graphics]
# For basic Pillow image perturbations
pip install nrtk[Pillow]
# For pybsm's sensor-based perturbations
pip install nrtk[pybsm]
```

**Note**: Choose either `graphics` or `headless` for OpenCV, not both.

More information on extras and related perturbers, including a complete list of
extras, can be found
[here](https://nrtk.readthedocs.io/en/latest/getting_started/installation.html#extras).

Details on the perturbers and their dependencies can be found
[here](https://nrtk.readthedocs.io/en/latest/reference/api/implementations.html).

For more detailed installation instructions, visit the
[installation documentation](https://nrtk.readthedocs.io/en/latest/getting_started/installation.html).

<!-- :auto installation: -->

<!-- :auto getting-started: -->

## Getting Started

Explore usage examples of the `nrtk` package in various contexts using the
Jupyter notebooks provided in the `./docs/examples/` directory.

<!-- :auto getting-started: -->

## Example: A First Look at NRTK Perturbations

Via the pyBSM package, NRTK exposes a large set of Optical Transfer Functions
(OTFs). These OTFs can simulate different environmental and sensor-based
effects. For example, the :ref:`JitterPerturber <JitterPerturber>` simulates
different levels of sensor jitter. By modifying its input parameters, you can
observe how sensor jitter affects image quality.

#### Input Image

Below is an example of an input image that will undergo a Jitter OTF
perturbation. This image represents the initial state before any transformation.

![augmented_image](https://raw.githubusercontent.com/Kitware/nrtk/main/docs/images/input.jpg)

#### Code Sample

Below is some example code that applies a Jitter OTF transformation::

```
from nrtk.impls.perturb_image.optical.otf import JitterPerturber
import numpy as np
from PIL import Image

INPUT_IMG_FILE = 'docs/images/input.jpg'
image = np.array(Image.open(INPUT_IMG_FILE))

otf = JitterPerturber(sx=8e-6, sy=8e-6, name="test_name")
out_image = otf(image)
```

This code uses default values and provides a sample input image. However, you
can adjust the parameters and use your own image to visualize the perturbation.
The sx and sy parameters (the root-mean-squared jitter amplitudes in radians, in
the x and y directions) are the primary way to customize a jitter perturber.
Larger jitter amplitude generate a larger Gaussian blur kernel.

#### Resulting Image

The output image below shows the effects of the Jitter OTF on the original
input. This result illustrates the Gaussian blur introduced due to simulated
sensor jitter.

![augmented_image](https://raw.githubusercontent.com/Kitware/nrtk/main/docs/images/output-jitter.jpg)

<!-- :auto documentation: -->

## Documentation

Documentation for both release snapshots and the latest main branch is available
on [ReadTheDocs](https://nrtk.readthedocs.io).

To build the Sphinx-based documentation locally for the latest reference:

```bash
# Install dependencies
poetry sync --with main,linting,tests,docs
# Navigate to the documentation root
cd docs
# Build the documentation
poetry run make html
# Open the generated documentation in your browser
firefox _build/html/index.html
```

<!-- :auto documentation: -->

<!-- :auto contributing: -->

## Contributing

Contributions are encouraged!

The following points help ensure contributions follow development practices.

- Follow the
  [JATIC Design Principles](https://cdao.pages.jatic.net/public/program/design-principles/).
- Adopt the Git Flow branching strategy.
- See the
  [release process documentation](https://nrtk.readthedocs.io/en/latest/development/release_process.html)
  for detailed release information.
- Additional contribution guidelines and issue reporting steps can be found in
  [CONTRIBUTING.md](https://github.com/Kitware/nrtk/blob/main/CONTRIBUTING.md).

<!-- :auto contributing: -->

<!-- :auto developer-tools: -->

### Developer Tools

Ensure the source tree is acquired locally before proceeding.

#### Poetry Install

You can install using [Poetry](https://python-poetry.org/):

> [!IMPORTANT] NRTK requires Poetry 2.2 or higher.

> [!WARNING] Users unfamiliar with Poetry should use caution. See
> [installation documentation](https://nrtk.readthedocs.io/en/latest/getting_started/installation.html#from-source)
> for more information.

```bash
poetry install --with main,linting,tests,docs --extras "<extra1> <extra2> ..."
```

#### Pre-commit Hooks

Pre-commit hooks ensure that code complies with required linting and formatting
guidelines. These hooks run automatically before commits but can also be
executed manually. To bypass checks during a commit, use the `--no-verify` flag.

To install and use pre-commit hooks:

```bash
# Install required dependencies
poetry sync --with main,linting,tests,docs
# Initialize pre-commit hooks for the repository
poetry run pre-commit install
# Run pre-commit checks on all files
poetry run pre-commit run --all-files
```

<!-- :auto developer-tools: -->

## NRTK Demonstration Tool

This [associated project](https://github.com/Kitware/nrtk-explorer) provides a
local web application that provides a demonstration of visual saliency
generation in a user interface. This provides an example of how image
perturbation, as generated by this package, can be utilized in a user interface
to facilitate dataset exploration. This tool uses the
[trame framework](https://kitware.github.io/trame/).

![image1](https://raw.githubusercontent.com/Kitware/nrtk/main/docs/figures/nrtk-explorer-example.png)

<!-- :auto license: -->

## License

[Apache 2.0](https://github.com/Kitware/nrtk/blob/main/LICENSE)

<!-- :auto license: -->

<!-- :auto contacts: -->

## Contacts

**Current Maintainers**: Brandon RichardWebster (@bjrichardwebster), Emily
Veenhuis (@eveenhuis)

We welcome contributions to NRTK! Please start discussions by opening an issue
or pull request on GitHub. This keeps the conversation visible and helps the
whole community benefit. Our preferred channels are public, but if you'd like to
reach out privately first, feel free to contact us at nrtk@kitware.com.

<!-- :auto contacts: -->

<!-- :auto acknowledgment: -->

## Acknowledgment

This material is based upon work supported by the Chief Digital and Artificial
Intelligence Office under Contract No. 519TC-23-9-2032. The views and
conclusions contained herein are those of the author(s) and should not be
interpreted as necessarily representing the official policies or endorsements,
either expressed or implied, of the U.S. Government.

<!-- :auto acknowledgment: -->