pytest-brightest 0.1.0
Bright ideas for improving your pytest experience
Requires Python: >=3.11
Keywords: llm, pytest, testing

# pytest-brightest

What is `pytest-brightest`? It is a `pytest` plugin that uses the brightest
ideas in software testing research to illuminate problems with your test suite
and brighten up the execution of your tests.

## Table of Contents

- [Perspectives](#perspectives)
- [Why `pytest-brightest`?](#why-pytest-brightest)
- [Requirements](#requirements)
- [Installation](#installation)
  - [Using `uv`](#using-uv)
  - [Using `poetry`](#using-poetry)
- [Modes of Operation](#modes-of-operation)
  - [Reordering](#reordering)
  - [Shuffling](#shuffling)
- [Command-Line Arguments](#command-line-arguments)
  - [Examples](#examples)
- [Contributing and Reporting Issues](#contributing-and-reporting-issues)

## Perspectives

`pytest-brightest` is a tool that helps software engineers run their test
suites more effectively. Here are three perspectives on how it can be useful:

- **The Busy Developer**: "I need to get feedback from my tests as quickly as
possible. By using `pytest-brightest` to run the fastest tests first, I can find
out if I've introduced any simple bugs in seconds, rather than waiting minutes
for the whole suite to run."
- **The Quality Assurance Engineer**: "My team is struggling with flaky tests
that pass or fail unpredictably. `pytest-brightest`'s shuffling capabilities
help us shine a light on these tests by breaking hidden dependencies, making our
test suite more reliable."
- **The Open-Source Maintainer**: "When a contributor submits a pull request, I
want to be confident it doesn't re-introduce old bugs. Reordering tests to run
previously failing ones first gives me a faster signal, streamlining my review
process."

## Why `pytest-brightest`?

In software testing, not all test executions are created equal. Some tests are
long, others are short. Some are prone to failure, while others are steadfastly
reliable. `pytest-brightest` helps you shine a light on these characteristics
to run your test suite more intelligently. It's not just about leveraging the
brightest ideas from software testing research; it's about making your `test`s
brighter (yes, the pun is intended!). Whether you want to get feedback faster,
find flaky tests, or simply bring more clarity to your testing chaos,
`pytest-brightest` is here to help.

## Requirements

To use `pytest-brightest`, you'll need:

- `pytest`
- `pytest-json-report`

`pytest-brightest` uses the data generated by `pytest-json-report` to work its
magic. If you have `pytest-json-report` installed, `pytest-brightest` will
automatically configure it for you.

## Installation

### Using `uv`

To install `pytest-brightest` using `uv`, a fast Python package installer and
resolver, run the following command:

```bash
uv add pytest-brightest --group dev
```

### Using `poetry`

To install `pytest-brightest` using `poetry`, a tool for dependency management
and packaging in Python, run the following command:

```bash
poetry add pytest-brightest --group dev
```

## Modes of Operation

`pytest-brightest` is enabled with the `--brightest` flag and can operate in
several modes, configured through the `--reorder-by-technique` option.

### Reordering

You can reorder your tests based on a variety of criteria:

- **`cost`**: Reorders tests based on their execution time from the previous
run. This is perfect for getting quick feedback by running faster tests first.
- **`name`**: Reorders tests alphabetically by their name.
- **`failure`**: Reorders tests based on their failure history, running tests
that failed in the previous run first. This is a powerful technique for
regression testing.

### Shuffling

- **`shuffle`**: Randomizes the order of your tests. This is an excellent way to
uncover hidden dependencies between tests and identify "flaky" tests that pass
or fail based on the order in which they are run.

## Command-Line Arguments

Here's a breakdown of the command-line arguments you can use to control
`pytest-brightest`:

- `--brightest`
    - **Description**: Enable the `pytest-brightest` plugin.
    - **Default**: `False`
- `--reorder-by-technique`
    - **Description**: The technique to use for reordering or shuffling tests.
    - **Options**: `shuffle`, `name`, `cost`, `failure`
    - **Default**: `None`
- `--reorder-by-focus`
    - **Description**: The scope of the reordering or shuffling.
    - **Options**: `modules-within-suite`, `tests-within-module`,
    `tests-across-modules`
    - **Default**: `tests-across-modules`
- `--reorder-in-direction`
    - **Description**: The direction of the reordering.
    - **Options**: `ascending`, `descending`
    - **Default**: `ascending`
- `--seed`
    - **Description**: A seed for the random number generator used for
    shuffling, ensuring reproducible results.
    - **Default**: (random)

### Examples

#### Run faster tests first

```bash
pytest --brightest --reorder-by-technique cost --reorder-in-direction ascending
```

#### Run previously failing tests first

```bash
pytest --brightest --reorder-by-technique failure --reorder-in-direction descending
```

#### Shuffle all tests with a specific seed

```bash
pytest --brightest --reorder-by-technique shuffle --seed 12345
```

#### Shuffle tests within each module, but keep the module order

```bash
pytest --brightest --reorder-by-technique shuffle --reorder-by-focus tests-within-module
```

## Contributing and Reporting Issues

Contributions are welcome! This project is open-source and thrives on community
involvement. Let's make testing brighter, together.

- **Issues**: If you find a bug or have a feature request, please [open an
issue](https://github.com/AstuteSource/pytest-brightest/issues).
- **Pull Requests**: If you'd like to contribute code, please [submit a pull
request](https://github.com/AstuteSource/pytest-brightest/pulls).