bmark-py 0.0.4
Benchmarking tool
License: MIT
Requires Python: >=3.5

# Bmark

Python benchmarking tool.

It's mostly used for benchmarking [FeatherStore](https://github.com/hakonmh/featherstore),
but can be used to benchmark any Python code.

## Installation

Bmark is available on [PyPI](https://pypi.org/project/bmark-py/):

```console
python -m pip install bmark-py
```

## Basic usage

First let's setup a class to be benchmarked, all methods except `run()` are
optional.

```python
import os
import bmark
import pandas as pd

read_bench = bmark.Benchmark()

@read_bench()  # Remember the parantheses
class read_csv(bmark.Benched):

    def __init__(self, shape, engine):
        self.name = f'pd.read_csv(engine={engine})'
        self.rows, self.cols = shape
        self._path = '_benchmarks'
        self.file_path = os.path.join(self._path, 'table.csv')
        self.engine = engine
        super().__init__()

    def run(self):
        """Code to be benchmarked"""
        pd.read_csv(self.file_path, engine=self.engine)

    def setup(self):
        """Setup for the entire duration of the timer"""
        data = {f'c{i}': range(self.rows) for i in range(self.cols)}
        self.df = pd.DataFrame(data)
        if not os.path.exists(self._path):
            os.makedirs(self._path)

    def teardown(self):
        """Final teardown after all the timings are done"""
        os.rmdir(self._path)

    def __enter__(self):
        """Called before each loop in the timer"""
        self.df.to_csv(self.file_path)
        return self  # Important

    def __exit__(self, *args):
        """Called after each loop in the timer"""
        os.remove(self.file_path)
```

We initialize a benchmark with `bmark.Benchmark()`. We can the register classes
to be benchmarked by using the `Benchmark` object as a decorator (as show above).

Each time we initialize a registered class it'll get added as an item to be
benchmarked:

```python
shape = (100_000, 10)
read_csv(shape, engine='c')
read_csv(shape, engine='python')
read_csv(shape, engine='pyarrow')

header = f'Read CSV benchmark {shape}'
read_bench.run(header, r=5, n=5, sort=True)

>>                 Read CSV benchmark (100000, 10)
 Name                        │ Hits │    Best │   Worst │ Comparison
─────────────────────────────┼──────┼─────────┼─────────┼────────────
 pd.read_csv(engine=pyarrow) │   25 │ 16.2 ms │ 20.5 ms │      1.00x
 pd.read_csv(engine=c)       │   25 │ 74.2 ms │  106 ms │      4.58x
 pd.read_csv(engine=python)  │   25 │  803 ms │  862 ms │     49.54x

Runtimes: total 46.5 s, benchmark 23.4 s, other 23.2 s
```

We can also populate benchmarks by passing all the objects we want benchmarked
in a list directly into the `bmark.Benchmark` constructor:

```python
shape = (500_000, 20)
items = (
    read_csv(shape, engine='c'),
    read_csv(shape, engine='python'),
    read_csv(shape, engine='pyarrow')
)
header = f'Read CSV benchmark {shape}'
bmark.Benchmark(items).run(header, r=1, n=5, sort=True)

>>           Read CSV benchmark (500000, 20)
 Name                        │ Hits │   Time │ Comparison
─────────────────────────────┼──────┼────────┼────────────
 pd.read_csv(engine=pyarrow) │    5 │ 112 ms │      1.00x
 pd.read_csv(engine=c)       │    5 │ 655 ms │      5.85x
 pd.read_csv(engine=python)  │    5 │ 7.49 s │     66.98x

Runtimes: total 1min 21s, benchmark 41.3 s, other 40.2 s
```

For a more in-depth example, see the FeatherStore
[benchmarking suite](https://github.com/hakonmh/featherstore/tree/master/benchmarks).