Module Import alias Import alias Import alias Assign NameStore TupleLoad Str Expr NameLoad FunctionDef arguments args NameParam defaults body Assign NameStore Call AttributeLoad NameLoad attr Str Expr Call AttributeLoad AttributeLoad NameLoad attr attr ListLoad Str Str Str Str Str Str Str Str Str Str Str Str Str Str Str Str Str Str Str Str Str Str Str Str Str Str Str Str Str Str Str Str Str Str Str Str Str Str Str Str Str Str Str Str Str Str Str Str Str Str Str Str Str Str Str Str Str Str Str Str Str Str Str Str Str Str Str Str Str Str Str Str Str Str Str Str decorator_list FunctionDef arguments args NameParam defaults body Assign NameStore Call AttributeLoad NameLoad attr Str Assign NameStore Call AttributeLoad NameLoad attr Str Str NameLoad Expr Call AttributeLoad AttributeLoad NameLoad attr attr ListLoad Str Str Str Str Str Str Str Str Str Str Str Str Str decorator_list Assign NameStore Str Assign NameStore Str Assign NameStore Str FunctionDef arguments args NameParam defaults body Assign NameStore Call AttributeLoad NameLoad attr keyword NameLoad Assign NameStore Call AttributeLoad NameLoad attr keyword NameLoad Assign NameStore Call AttributeLoad NameLoad attr Str Str Str NameLoad NameLoad Expr Call AttributeLoad AttributeLoad NameLoad attr attr ListLoad Str Str Str Str Str Str Str Str Str Str Str Str Str Str Str Str Str Str Str Str Str Str Str Str Str Str Str Str Str Str Str Str decorator_list FunctionDef arguments args NameParam defaults body Assign NameStore Call AttributeLoad NameLoad attr keyword NameLoad Assign NameStore Call AttributeLoad NameLoad attr keyword NameLoad Assign NameStore Call AttributeLoad NameLoad attr Str Str Str NameLoad NameLoad Expr Call AttributeLoad AttributeLoad NameLoad attr attr ListLoad Str Str Str Str Str Str Str Str Str Str Str Str Str Str Str Str Str Str Str Str Str Str Str Str decorator_list FunctionDef arguments args NameParam defaults body Assign NameStore Call AttributeLoad NameLoad attr keyword NameLoad Assign NameStore Call AttributeLoad NameLoad attr keyword NameLoad Assign NameStore Call AttributeLoad NameLoad attr Str Str Str NameLoad NameLoad Expr Call AttributeLoad AttributeLoad NameLoad attr attr ListLoad Str Str Str Str Str Str Str Str Str Str Str Str Str Str Str Str Str Str Str Str Str Str Str Str Str Str Str Str Str Str Str Str Str Str Str decorator_list FunctionDef arguments args NameParam defaults body Assign NameStore Call AttributeLoad NameLoad attr keyword NameLoad Assign NameStore Call AttributeLoad NameLoad attr keyword NameLoad Assign NameStore Call AttributeLoad NameLoad attr Str Str Str NameLoad NameLoad Expr Call AttributeLoad AttributeLoad NameLoad attr attr ListLoad Str Str Str Str Str Str Str Str Str Str Str Str Str Str Str Str Str Str Str Str Str Str Str Str Str decorator_list FunctionDef arguments args NameParam defaults body Assign NameStore Call AttributeLoad NameLoad attr keyword NameLoad Assign NameStore Call AttributeLoad NameLoad attr Str Str Str Str Str NameLoad Expr Call AttributeLoad AttributeLoad NameLoad attr attr ListLoad Str Str Str Str Str Str Str Str Str Str Str Str Str Str Str Str Str Str Str Str Str Str Str Str Str decorator_list FunctionDef arguments args NameParam defaults body Assign NameStore Call AttributeLoad NameLoad attr keyword NameLoad Assign NameStore Call AttributeLoad NameLoad attr Str Str Str Str Str NameLoad Expr Call AttributeLoad AttributeLoad NameLoad attr attr ListLoad Str Str Str Str Str Str Str Str Str Str Str Str Str Str Str Str Str Str Str Str Str Str Str Str Str Str Str Str Str Str Str Str Str Str Str decorator_list FunctionDef arguments args NameParam defaults body Assign NameStore Call AttributeLoad NameLoad attr keyword NameLoad Assign NameStore Call AttributeLoad NameLoad attr keyword NameLoad Assign NameStore Call AttributeLoad NameLoad attr Str Str Str NameLoad NameLoad Expr Call AttributeLoad AttributeLoad NameLoad attr attr ListLoad Str Str Str Str Str Str Str Str Str decorator_list FunctionDef arguments args NameParam defaults body Assign NameStore Call AttributeLoad NameLoad attr Str Assign NameStore Call AttributeLoad NameLoad attr NameLoad Str Expr Call AttributeLoad AttributeLoad NameLoad attr attr ListLoad Str Str decorator_list FunctionDef arguments args NameParam defaults body Assign NameStore Call AttributeLoad NameLoad attr NameLoad Assign NameStore Call AttributeLoad NameLoad attr Str Str NameLoad Expr Call AttributeLoad AttributeLoad NameLoad attr attr ListLoad Str decorator_list FunctionDef arguments args NameParam defaults body Assign NameStore Call AttributeLoad NameLoad attr NameLoad Assign NameStore Call AttributeLoad NameLoad attr Str Str NameLoad Expr Call AttributeLoad AttributeLoad NameLoad attr attr ListLoad Str decorator_list FunctionDef arguments args NameParam defaults body Assign NameStore Call AttributeLoad NameLoad attr NameLoad Assign NameStore Call AttributeLoad NameLoad attr Str Str Str NameLoad Expr Call AttributeLoad AttributeLoad NameLoad attr attr ListLoad Str Str Str Str Str Str Str Str Str decorator_list FunctionDef arguments args NameParam defaults body Assign NameStore Call AttributeLoad NameLoad attr NameLoad Assign NameStore Call AttributeLoad NameLoad attr Str Str NameLoad Expr Call AttributeLoad AttributeLoad NameLoad attr attr ListLoad Str Str Str Str Str Str Str Str Str decorator_list FunctionDef arguments args NameParam defaults body Assign NameStore Call AttributeLoad NameLoad attr NameLoad Assign NameStore Call AttributeLoad NameLoad attr Str Str NameLoad Expr Call AttributeLoad AttributeLoad NameLoad attr attr ListLoad Str Str decorator_list FunctionDef arguments args NameParam defaults body Assign NameStore Call AttributeLoad NameLoad attr NameLoad Assign NameStore Call AttributeLoad NameLoad attr Str Str Str NameLoad Expr Call AttributeLoad AttributeLoad NameLoad attr attr ListLoad Str decorator_list Call AttributeLoad AttributeLoad NameLoad attr attr Str FunctionDef arguments args NameParam defaults body Assign NameStore Call AttributeLoad NameLoad attr NameLoad Assign NameStore Call AttributeLoad NameLoad attr Str Str NameLoad Expr Call AttributeLoad AttributeLoad NameLoad attr attr ListLoad Str Str decorator_list FunctionDef arguments args NameParam defaults body Assign NameStore Call AttributeLoad NameLoad attr NameLoad Assign NameStore Call AttributeLoad NameLoad attr Str Str NameLoad Expr Call AttributeLoad AttributeLoad NameLoad attr attr ListLoad Str Str decorator_list FunctionDef arguments args NameParam defaults body Assign NameStore Call AttributeLoad NameLoad attr NameLoad Assign NameStore Call AttributeLoad NameLoad attr Str Str NameLoad Expr Call AttributeLoad AttributeLoad NameLoad attr attr ListLoad Str Str decorator_list FunctionDef arguments args NameParam defaults body Assign NameStore Call AttributeLoad NameLoad attr NameLoad Assign NameStore Call AttributeLoad NameLoad attr Str Str NameLoad Expr Call AttributeLoad AttributeLoad NameLoad attr attr ListLoad Str Str decorator_list FunctionDef arguments args NameParam defaults body Assign NameStore Call AttributeLoad NameLoad attr NameLoad Assign NameStore Call AttributeLoad NameLoad attr Str Str NameLoad Expr Call AttributeLoad AttributeLoad NameLoad attr attr ListLoad Str Str decorator_list FunctionDef arguments args NameParam defaults body Assign NameStore Call AttributeLoad NameLoad attr NameLoad Expr Call AttributeLoad NameLoad attr Str Str Str NameLoad Assign NameStore Call AttributeLoad NameLoad attr Str Str Str Str NameLoad Expr Call AttributeLoad AttributeLoad NameLoad attr attr ListLoad Str Assign NameStore Call AttributeLoad NameLoad attr Str Str Str Str NameLoad Expr Call AttributeLoad AttributeLoad NameLoad attr attr ListLoad Str decorator_list FunctionDef arguments args NameParam defaults body Assign NameStore Call AttributeLoad NameLoad attr NameLoad Expr Call AttributeLoad NameLoad attr Str Str Str NameLoad Assign NameStore Call AttributeLoad NameLoad attr Str Str Str Str NameLoad Expr Call AttributeLoad AttributeLoad NameLoad attr attr ListLoad Str Assign NameStore Call AttributeLoad NameLoad attr Str Str Str Str NameLoad Expr Call AttributeLoad AttributeLoad NameLoad attr attr ListLoad Str decorator_list FunctionDef arguments args NameParam defaults body Assign NameStore Call AttributeLoad NameLoad attr NameLoad Expr Call AttributeLoad NameLoad attr Str Str Str NameLoad Assign NameStore Call AttributeLoad NameLoad attr Str Str Str Str NameLoad Expr Call AttributeLoad AttributeLoad NameLoad attr attr ListLoad Str decorator_list FunctionDef arguments args NameParam defaults body Assign NameStore Call AttributeLoad NameLoad attr NameLoad Expr Call AttributeLoad NameLoad attr Str Str Str NameLoad Assign NameStore Call AttributeLoad NameLoad attr Str Str Str NameLoad Str Expr Call AttributeLoad AttributeLoad NameLoad attr attr ListLoad Str decorator_list FunctionDef arguments args NameParam defaults body Assign NameStore Call AttributeLoad NameLoad attr NameLoad Assign NameStore Call AttributeLoad NameLoad attr Str Str Str NameLoad Str Expr Call AttributeLoad AttributeLoad NameLoad attr attr ListLoad Str decorator_list FunctionDef arguments args NameParam defaults body Assign NameStore Call AttributeLoad NameLoad attr NameLoad Assign NameStore Call AttributeLoad NameLoad attr Str Str NameLoad Str Str Expr Call AttributeLoad AttributeLoad NameLoad attr attr ListLoad Str decorator_list FunctionDef arguments args NameParam defaults body Assign NameStore Call AttributeLoad NameLoad attr NameLoad Assign NameStore Call AttributeLoad NameLoad attr Str Str Str NameLoad Str Expr Call AttributeLoad AttributeLoad NameLoad attr attr ListLoad Str decorator_list FunctionDef arguments args NameParam defaults body Assign NameStore Call AttributeLoad NameLoad attr NameLoad Assign NameStore Call AttributeLoad NameLoad attr Str Str NameLoad Str Str Expr Call AttributeLoad AttributeLoad NameLoad attr attr ListLoad Str decorator_list FunctionDef arguments args NameParam defaults body Assign NameStore Call AttributeLoad NameLoad attr NameLoad Assign NameStore Call AttributeLoad NameLoad attr Str Str Str Str NameLoad Expr Call AttributeLoad AttributeLoad NameLoad attr attr ListLoad Str Str Str Str Str decorator_list FunctionDef arguments args NameParam defaults body Assign NameStore Call AttributeLoad NameLoad attr NameLoad Assign NameStore Call AttributeLoad NameLoad attr Str Str Str NameLoad Expr Call AttributeLoad AttributeLoad NameLoad attr attr ListLoad Str Expr Call AttributeLoad NameLoad attr Call AttributeLoad Call AttributeLoad SubscriptLoad Call AttributeLoad Call AttributeLoad AttributeLoad NameLoad attr attr Str attr Index Num attr Str attr decorator_list FunctionDef arguments args NameParam defaults body Assign NameStore Call AttributeLoad NameLoad attr NameLoad Assign NameStore Call AttributeLoad NameLoad attr Str Str Str NameLoad Expr Call AttributeLoad AttributeLoad NameLoad attr attr ListLoad Str Assert CompareEq ListComp AttributeLoad NameLoad attr comprehension NameStore Call AttributeLoad AttributeLoad NameLoad attr attr Str keyword NameLoad ListLoad Str decorator_list FunctionDef arguments args NameParam defaults body Assign NameStore Call AttributeLoad NameLoad attr NameLoad Assign NameStore Call AttributeLoad NameLoad attr Str Str Str NameLoad Expr Call AttributeLoad AttributeLoad NameLoad attr attr ListLoad Str Expr Call AttributeLoad NameLoad attr Call AttributeLoad SubscriptLoad Call AttributeLoad SubscriptLoad Call AttributeLoad Call AttributeLoad AttributeLoad NameLoad attr attr Str attr Index Num attr Str Index Num attr decorator_list FunctionDef arguments args NameParam defaults body Assign NameStore Call AttributeLoad NameLoad attr NameLoad Assign NameStore Call AttributeLoad NameLoad attr Str Str NameLoad Expr Call AttributeLoad AttributeLoad NameLoad attr attr ListLoad Str Str decorator_list FunctionDef arguments args NameParam defaults body Assign NameStore Call AttributeLoad NameLoad attr NameLoad Assign NameStore Call AttributeLoad NameLoad attr Str NameLoad Expr Call AttributeLoad AttributeLoad NameLoad attr attr ListLoad Str Str Str Str Str Str Str Str Str decorator_list FunctionDef arguments args NameParam defaults body Assign NameStore Call AttributeLoad NameLoad attr NameLoad Assign NameStore Call AttributeLoad NameLoad attr Str NameLoad Expr Call AttributeLoad AttributeLoad NameLoad attr attr ListLoad Str Str Str Str Str Str Str Str Str decorator_list FunctionDef arguments args NameParam defaults body Assign NameStore Call AttributeLoad NameLoad attr NameLoad Assign NameStore Call AttributeLoad NameLoad attr Str NameLoad Expr Call AttributeLoad AttributeLoad NameLoad attr attr ListLoad Str Str decorator_list FunctionDef arguments args NameParam defaults body Assign NameStore Call AttributeLoad NameLoad attr NameLoad Assign NameStore Call AttributeLoad NameLoad attr Str NameLoad Expr Call AttributeLoad AttributeLoad NameLoad attr attr ListLoad Str Str Str Str Str Str Str Str Str decorator_list FunctionDef arguments args NameParam defaults body Assign NameStore Call AttributeLoad NameLoad attr NameLoad Assign NameStore Call AttributeLoad NameLoad attr Str NameLoad Expr Call AttributeLoad AttributeLoad NameLoad attr attr ListLoad Str Str decorator_list FunctionDef arguments args NameParam defaults body Expr Call AttributeLoad NameLoad attr Str Assign NameStore Call AttributeLoad NameLoad attr NameLoad Assign NameStore Call AttributeLoad NameLoad attr Str Str Str Str NameLoad Expr Call AttributeLoad AttributeLoad NameLoad attr attr ListLoad Str decorator_list FunctionDef arguments args NameParam defaults body Expr Call AttributeLoad NameLoad attr Str Assign NameStore Call AttributeLoad NameLoad attr NameLoad Assign NameStore Call AttributeLoad NameLoad attr Str Str Str Str NameLoad Expr Call AttributeLoad AttributeLoad NameLoad attr attr ListLoad Str Str Str decorator_list FunctionDef arguments args NameParam defaults body Expr Str Assign NameStore Call AttributeLoad NameLoad attr Str Assign NameStore Call AttributeLoad NameLoad attr Str NameLoad Expr Call AttributeLoad AttributeLoad NameLoad attr attr ListLoad Str Str Str Str Str Str Str Str Str Str Str Str Str Str Str Str Str Str Str Str Str Str Str Str Str Str Str Str Str Str Str Str Str Str Str Str Str Str Str Str Str Str Str Str Str Str Str Str Str Str Str Str Str Str Str Str Str Str decorator_list Assign NameStore Str FunctionDef arguments args NameParam defaults body Assign NameStore Call AttributeLoad NameLoad attr NameLoad Assign NameStore Call AttributeLoad NameLoad attr Str Str NameLoad Expr Call AttributeLoad AttributeLoad NameLoad attr attr ListLoad Str Str Str Str Str Str Str Str Str Str Str Str Str Str Str Str Str decorator_list FunctionDef arguments args NameParam defaults body Assign NameStore Call AttributeLoad NameLoad attr NameLoad Assign NameStore Call AttributeLoad NameLoad attr Str Str Str NameLoad Expr Call AttributeLoad AttributeLoad NameLoad attr attr ListLoad Str Str Str Str Str Str Str decorator_list FunctionDef arguments args NameParam defaults body Assign NameStore Call AttributeLoad NameLoad attr NameLoad Assign NameStore Call AttributeLoad NameLoad attr Str Str Str NameLoad Expr Call AttributeLoad AttributeLoad NameLoad attr attr ListLoad Str Str Str Str Str Str Str decorator_list FunctionDef arguments args NameParam defaults body Assign NameStore Call AttributeLoad NameLoad attr NameLoad Assign NameStore Call AttributeLoad NameLoad attr Str Str Str Str NameLoad Expr Call AttributeLoad AttributeLoad NameLoad attr attr ListLoad Str Str Str Str Str Str Str Str Str decorator_list FunctionDef arguments args NameParam defaults body Assign NameStore Call AttributeLoad NameLoad attr NameLoad Assign NameStore Call AttributeLoad NameLoad attr Str Str Str NameLoad Expr Call AttributeLoad AttributeLoad NameLoad attr attr ListLoad Str Str Str Str Str Str Str Str Str Str Str Str Str Str Str decorator_list FunctionDef arguments args NameParam defaults body Assign NameStore Call AttributeLoad NameLoad attr NameLoad Assign NameStore Call AttributeLoad NameLoad attr Str Str NameLoad Expr Call AttributeLoad AttributeLoad NameLoad attr attr ListLoad Str Str Str Str Str decorator_list
N/A N/A json N/A platform N/A pytest N/A pytest_plugins N/A pytester N/A platform test_help N/A N/A testdir N/A N/A N/A result N/A N/A testdir runpytest --help N/A N/A N/A N/A result stdout fnmatch_lines N/A * * benchmark:   --benchmark-min-time=SECONDS                         Minimum time per round in seconds. Default: '0.000005'   --benchmark-max-time=SECONDS                         Maximum run time per test - it will be repeated until                         this total time is reached. It may be exceeded if test                         function is very slow or --benchmark-min-rounds is                         large (it takes precedence). Default: '1.0'   --benchmark-min-rounds=NUM                         Minimum rounds, even if total time would exceed                         `--max-time`. Default: 5   --benchmark-timer=FUNC                         Timer to use when measuring time. Default:*   --benchmark-calibration-precision=NUM                         Precision to use when calibrating number of                         iterations. Precision of 10 will make the timer look                         10 times more accurate, at a cost of less precise                         measure of deviations. Default: 10   --benchmark-warmup=[KIND]                         Activates warmup. Will run the test function up to                         number of times in the calibration phase. See                         `--benchmark-warmup-iterations`. Note: Even the warmup                         phase obeys --benchmark-max-time. Available KIND:                         'auto', 'off', 'on'. Default: 'auto' (automatically                         activate on PyPy).   --benchmark-warmup-iterations=NUM                         Max number of iterations to run in the warmup phase.                         Default: 100000   --benchmark-disable-gc                         Disable GC during benchmarks.   --benchmark-skip      Skip running any tests that contain benchmarks.   --benchmark-only      Only run benchmarks.   --benchmark-save=NAME                         Save the current run into 'STORAGE-                         PATH/counter_NAME.json'.   --benchmark-autosave  Autosave the current run into 'STORAGE-                         PATH/counter_*.json   --benchmark-save-data                         Use this to make --benchmark-save and --benchmark-                         autosave include all the timing data, not just the                         stats.   --benchmark-json=PATH                         Dump a JSON report into PATH. Note that this will                         include the complete data (all the timings, not just                         the stats).   --benchmark-compare=[NUM]                         Compare the current run against run NUM or the latest                         saved run if unspecified.   --benchmark-compare-fail=EXPR=[EXPR=...]                         Fail test if performance regresses according to given                         EXPR (eg: min:5% or mean:0.001 for number of seconds).                         Can be used multiple times.   --benchmark-storage=STORAGE-PATH                         Specify a different path to store the runs (when                         --benchmark-save or --benchmark-autosave are used).                         Default: './.benchmarks'   --benchmark-verbose   Dump diagnostic and progress information.   --benchmark-sort=COL  Column to sort on. Can be one of: 'min', 'max',                         'mean', 'stddev', 'name', 'fullname'. Default: 'min'   --benchmark-group-by=LABEL                         How to group tests. Can be one of: 'group', 'name',                         'fullname', 'func', 'fullfunc', 'param' or                         'param:NAME', where NAME is the name passed to                         @pytest.parametrize. Default: 'group'   --benchmark-columns=LABELS                         Comma-separated list of columns to show in the result                         table. Default: 'min, max, mean, stddev, median, iqr,                         outliers, rounds, iterations'   --benchmark-histogram=[FILENAME-PREFIX]                         Plot graphs of min/max/avg/stddev over time in                         FILENAME-PREFIX-test_name.svg. If FILENAME-PREFIX                         contains slashes ('/') then directories will be                         created. Default: '*' * N/A test_groups N/A N/A testdir N/A N/A N/A test N/A N/A testdir makepyfile """
    >>> print('Yay, doctests!')
    Yay, doctests!
"""
import time
import pytest

def test_fast(benchmark):
    benchmark(lambda: time.sleep(0.000001))
    assert 1 == 1

def test_slow(benchmark):
    benchmark(lambda: time.sleep(0.001))
    assert 1 == 1

@pytest.mark.benchmark(group="A")
def test_slower(benchmark):
    benchmark(lambda: time.sleep(0.01))
    assert 1 == 1

@pytest.mark.benchmark(group="A", warmup=True)
def test_xfast(benchmark):
    benchmark(lambda: None)
    assert 1 == 1
 N/A result N/A N/A testdir runpytest -vv --doctest-modules test N/A N/A N/A N/A result stdout fnmatch_lines N/A *collected 5 items * test_groups.py::*test_groups PASSED test_groups.py::test_fast PASSED test_groups.py::test_slow PASSED test_groups.py::test_slower PASSED test_groups.py::test_xfast PASSED * * benchmark: 2 tests * * * benchmark 'A': 2 tests * * *====== 5 passed* seconds ======* N/A N/A SIMPLE_TEST 
"""
    >>> print('Yay, doctests!')
    Yay, doctests!
"""
import time
import pytest

def test_fast(benchmark):
    @benchmark
    def result():
        return time.sleep(0.000001)
    assert result == None

def test_slow(benchmark):
    benchmark(lambda: time.sleep(0.1))
    assert 1 == 1
 N/A GROUPING_TEST 
import pytest

@pytest.mark.parametrize("foo", range(2))
@pytest.mark.benchmark(group="A")
def test_a(benchmark, foo):
    benchmark(str)

@pytest.mark.parametrize("foo", range(2))
@pytest.mark.benchmark(group="B")
def test_b(benchmark, foo):
    benchmark(int)
 N/A GROUPING_PARAMS_TEST 
import pytest

@pytest.mark.parametrize("bar", ["bar1", "bar2"])
@pytest.mark.parametrize("foo", ["foo1", "foo2"])
@pytest.mark.benchmark(group="A")
def test_a(benchmark, foo, bar):
    benchmark(str)


@pytest.mark.parametrize("bar", ["bar1", "bar2"])
@pytest.mark.parametrize("foo", ["foo1", "foo2"])
@pytest.mark.benchmark(group="B")
def test_b(benchmark, foo, bar):
    benchmark(int)
 test_group_by_name N/A N/A testdir N/A N/A N/A test_x N/A N/A testdir makepyfile test_x GROUPING_TEST N/A test_y N/A N/A testdir makepyfile test_y GROUPING_TEST N/A result N/A N/A testdir runpytest --benchmark-max-time=0.0000001 --benchmark-group-by name test_x test_y N/A N/A N/A N/A result stdout fnmatch_lines N/A * * * * * * benchmark 'test_a[[]0[]]': 2 tests * Name (time in ?s)     * ----------------------* test_a[[]0[]]             * test_a[[]0[]]             * ----------------------* * * benchmark 'test_a[[]1[]]': 2 tests * Name (time in ?s)     * ----------------------* test_a[[]1[]]             * test_a[[]1[]]             * ----------------------* * * benchmark 'test_b[[]0[]]': 2 tests * Name (time in ?s)     * ----------------------* test_b[[]0[]]             * test_b[[]0[]]             * ----------------------* * * benchmark 'test_b[[]1[]]': 2 tests * Name (time in ?s)     * ----------------------* test_b[[]1[]]             * test_b[[]1[]]             * ----------------------* N/A test_group_by_func N/A N/A testdir N/A N/A N/A test_x N/A N/A testdir makepyfile test_x GROUPING_TEST N/A test_y N/A N/A testdir makepyfile test_y GROUPING_TEST N/A result N/A N/A testdir runpytest --benchmark-max-time=0.0000001 --benchmark-group-by func test_x test_y N/A N/A N/A N/A result stdout fnmatch_lines N/A * * * * * benchmark 'test_a': 4 tests * Name (time in ?s)     * ----------------------* test_a[[]*[]]             * test_a[[]*[]]             * test_a[[]*[]]             * test_a[[]*[]]             * ----------------------* * * benchmark 'test_b': 4 tests * Name (time in ?s)     * ----------------------* test_b[[]*[]]             * test_b[[]*[]]             * test_b[[]*[]]             * test_b[[]*[]]             * ----------------------* * * ============* 8 passed* seconds ============* N/A test_group_by_fullfunc N/A N/A testdir N/A N/A N/A test_x N/A N/A testdir makepyfile test_x GROUPING_TEST N/A test_y N/A N/A testdir makepyfile test_y GROUPING_TEST N/A result N/A N/A testdir runpytest --benchmark-max-time=0.0000001 --benchmark-group-by fullfunc test_x test_y N/A N/A N/A N/A result stdout fnmatch_lines N/A * * * * * * benchmark 'test_x.py::test_a': 2 tests * Name (time in ?s) * ------------------* test_a[[]*[]]         * test_a[[]*[]]         * ------------------* * benchmark 'test_x.py::test_b': 2 tests * Name (time in ?s) * ------------------* test_b[[]*[]]         * test_b[[]*[]]         * ------------------* * benchmark 'test_y.py::test_a': 2 tests * Name (time in ?s) * ------------------* test_a[[]*[]]         * test_a[[]*[]]         * ------------------* * benchmark 'test_y.py::test_b': 2 tests * Name (time in ?s) * ------------------* test_b[[]*[]]         * test_b[[]*[]]         * ------------------* (*) Outliers: 1 Standard Deviation from M* ============* 8 passed* seconds ============* N/A test_group_by_param_all N/A N/A testdir N/A N/A N/A test_x N/A N/A testdir makepyfile test_x GROUPING_TEST N/A test_y N/A N/A testdir makepyfile test_y GROUPING_TEST N/A result N/A N/A testdir runpytest --benchmark-max-time=0.0000001 --benchmark-group-by param test_x test_y N/A N/A N/A N/A result stdout fnmatch_lines N/A * * * * * * benchmark '0': 4 tests * Name (time in ?s)  * -------------------* test_*[[]0[]]          * test_*[[]0[]]          * test_*[[]0[]]          * test_*[[]0[]]          * -------------------* * benchmark '1': 4 tests * Name (time in ?s) * ------------------* test_*[[]1[]]         * test_*[[]1[]]         * test_*[[]1[]]         * test_*[[]1[]]         * ------------------* (*) Outliers: 1 Standard Deviation from Mean; 1.5 IQR (InterQuartile Range) from 1st Quartile and 3rd Quartile. ============* 8 passed* seconds ============* N/A test_group_by_param_select N/A N/A testdir N/A N/A N/A test_x N/A N/A testdir makepyfile test_x GROUPING_PARAMS_TEST N/A result N/A N/A testdir runpytest --benchmark-max-time=0.0000001 --benchmark-group-by param:foo --benchmark-sort fullname test_x N/A N/A N/A N/A result stdout fnmatch_lines N/A * * * * * * benchmark 'foo=foo1': 4 tests * Name (time in ?s)  * -------------------* test_a[[]foo1-bar1[]]    * test_a[[]foo1-bar2[]]    * test_b[[]foo1-bar1[]]    * test_b[[]foo1-bar2[]]    * -------------------* * benchmark 'foo=foo2': 4 tests * Name (time in ?s) * ------------------* test_a[[]foo2-bar1[]]    * test_a[[]foo2-bar2[]]    * test_b[[]foo2-bar1[]]    * test_b[[]foo2-bar2[]]    * ------------------* (*) Outliers: 1 Standard Deviation from Mean; 1.5 IQR (InterQuartile Range) from 1st Quartile and 3rd Quartile. ============* 8 passed* seconds ============* N/A test_group_by_param_select_multiple N/A N/A testdir N/A N/A N/A test_x N/A N/A testdir makepyfile test_x GROUPING_PARAMS_TEST N/A result N/A N/A testdir runpytest --benchmark-max-time=0.0000001 --benchmark-group-by param:foo,param:bar --benchmark-sort fullname test_x N/A N/A N/A N/A result stdout fnmatch_lines N/A * * * * * * benchmark 'foo=foo1 bar=bar1': 2 tests * Name (time in ?s)  * -------------------* test_a[[]foo1-bar1[]]    * test_b[[]foo1-bar1[]]    * -------------------* * benchmark 'foo=foo1 bar=bar2': 2 tests * Name (time in ?s)  * -------------------* test_a[[]foo1-bar2[]]    * test_b[[]foo1-bar2[]]    * -------------------* * benchmark 'foo=foo2 bar=bar1': 2 tests * Name (time in ?s) * ------------------* test_a[[]foo2-bar1[]]    * test_b[[]foo2-bar1[]]    * -------------------* * benchmark 'foo=foo2 bar=bar2': 2 tests * Name (time in ?s)  * -------------------* test_a[[]foo2-bar2[]]    * test_b[[]foo2-bar2[]]    * ------------------* (*) Outliers: 1 Standard Deviation from Mean; 1.5 IQR (InterQuartile Range) from 1st Quartile and 3rd Quartile. ============* 8 passed* seconds ============* N/A test_group_by_fullname N/A N/A testdir N/A N/A N/A test_x N/A N/A testdir makepyfile test_x GROUPING_TEST N/A test_y N/A N/A testdir makepyfile test_y GROUPING_TEST N/A result N/A N/A testdir runpytest --benchmark-max-time=0.0000001 --benchmark-group-by fullname test_x test_y N/A N/A N/A N/A result stdout fnmatch_lines_random N/A * benchmark 'test_x.py::test_a[[]0[]]': 1 tests * * benchmark 'test_x.py::test_a[[]1[]]': 1 tests * * benchmark 'test_x.py::test_b[[]0[]]': 1 tests * * benchmark 'test_x.py::test_b[[]1[]]': 1 tests * * benchmark 'test_y.py::test_a[[]0[]]': 1 tests * * benchmark 'test_y.py::test_a[[]1[]]': 1 tests * * benchmark 'test_y.py::test_b[[]0[]]': 1 tests * * benchmark 'test_y.py::test_b[[]1[]]': 1 tests * ============* 8 passed* seconds ============* N/A test_double_use N/A N/A testdir N/A N/A N/A test N/A N/A testdir makepyfile 
def test_a(benchmark):
    benchmark(lambda: None)
    benchmark.pedantic(lambda: None)

def test_b(benchmark):
    benchmark.pedantic(lambda: None)
    benchmark(lambda: None)
 N/A result N/A N/A testdir runpytest test --tb=line N/A N/A N/A N/A result stdout fnmatch_lines N/A *FixtureAlreadyUsed: Fixture can only be used once. Previously it was used in benchmark(...) mode. *FixtureAlreadyUsed: Fixture can only be used once. Previously it was used in benchmark.pedantic(...) mode. N/A test_conflict_between_only_and_skip N/A N/A testdir N/A N/A N/A test N/A N/A testdir makepyfile SIMPLE_TEST N/A result N/A N/A testdir runpytest --benchmark-only --benchmark-skip test N/A N/A N/A N/A result stderr fnmatch_lines N/A ERROR: Can't have both --benchmark-only and --benchmark-skip options. N/A test_conflict_between_only_and_disable N/A N/A testdir N/A N/A N/A test N/A N/A testdir makepyfile SIMPLE_TEST N/A result N/A N/A testdir runpytest --benchmark-only --benchmark-disable test N/A N/A N/A N/A result stderr fnmatch_lines N/A ERROR: Can't have both --benchmark-only and --benchmark-disable options. Note that --benchmark-disable is automatically activated if xdist is on or you're missing the statistics dependency. N/A test_max_time_min_rounds N/A N/A testdir N/A N/A N/A test N/A N/A testdir makepyfile SIMPLE_TEST N/A result N/A N/A testdir runpytest --doctest-modules --benchmark-max-time=0.000001 --benchmark-min-rounds=1 test N/A N/A N/A N/A result stdout fnmatch_lines N/A *collected 3 items test_max_time_min_rounds.py ... * benchmark: 2 tests * Name (time in ?s) * Min * Max * Mean * StdDev * Rounds * Iterations ------* test_fast          * 1  * test_slow          * 1  * ------* *====== 3 passed* seconds ======* N/A test_max_time N/A N/A testdir N/A N/A N/A test N/A N/A testdir makepyfile SIMPLE_TEST N/A result N/A N/A testdir runpytest --doctest-modules --benchmark-max-time=0.000001 test N/A N/A N/A N/A result stdout fnmatch_lines N/A *collected 3 items test_max_time.py ... * benchmark: 2 tests * Name (time in ?s) * Min * Max * Mean * StdDev * Rounds * Iterations ------* test_fast          * 5  * test_slow          * 5  * ------* *====== 3 passed* seconds ======* N/A test_bogus_max_time N/A N/A testdir N/A N/A N/A test N/A N/A testdir makepyfile SIMPLE_TEST N/A result N/A N/A testdir runpytest --doctest-modules --benchmark-max-time=bogus test N/A N/A N/A N/A result stderr fnmatch_lines N/A usage: py* [[]options[]] [[]file_or_dir[]] [[]file_or_dir[]] [[]...[]] py*: error: argument --benchmark-max-time: Invalid decimal value 'bogus': InvalidOperation* N/A test_pep418_timer N/A N/A testdir N/A N/A N/A test N/A N/A testdir makepyfile SIMPLE_TEST N/A result N/A N/A testdir runpytest --benchmark-max-time=0.0000001 --doctest-modules --benchmark-timer=pep418.perf_counter test N/A N/A N/A N/A result stdout fnmatch_lines N/A * (defaults: timer=*.perf_counter* N/A N/A N/A N/A pytest mark skipif platform.python_implementation() == 'PyPy' test_bad_save N/A N/A testdir N/A N/A N/A test N/A N/A testdir makepyfile SIMPLE_TEST N/A result N/A N/A testdir runpytest --doctest-modules --benchmark-save=asd:f? test N/A N/A N/A N/A result stderr fnmatch_lines N/A usage: py* [[]options[]] [[]file_or_dir[]] [[]file_or_dir[]] [[]...[]] py*: error: argument --benchmark-save: Must not contain any of these characters: /:*?<>|\ (it has ':?') N/A test_bad_save_2 N/A N/A testdir N/A N/A N/A test N/A N/A testdir makepyfile SIMPLE_TEST N/A result N/A N/A testdir runpytest --doctest-modules --benchmark-save= test N/A N/A N/A N/A result stderr fnmatch_lines N/A usage: py* [[]options[]] [[]file_or_dir[]] [[]file_or_dir[]] [[]...[]] py*: error: argument --benchmark-save: Can't be empty. N/A test_bad_compare_fail N/A N/A testdir N/A N/A N/A test N/A N/A testdir makepyfile SIMPLE_TEST N/A result N/A N/A testdir runpytest --doctest-modules --benchmark-compare-fail=? test N/A N/A N/A N/A result stderr fnmatch_lines N/A usage: py* [[]options[]] [[]file_or_dir[]] [[]file_or_dir[]] [[]...[]] py*: error: argument --benchmark-compare-fail: Could not parse value: '?'. N/A test_bad_rounds N/A N/A testdir N/A N/A N/A test N/A N/A testdir makepyfile SIMPLE_TEST N/A result N/A N/A testdir runpytest --doctest-modules --benchmark-min-rounds=asd test N/A N/A N/A N/A result stderr fnmatch_lines N/A usage: py* [[]options[]] [[]file_or_dir[]] [[]file_or_dir[]] [[]...[]] py*: error: argument --benchmark-min-rounds: invalid literal for int() with base 10: 'asd' N/A test_bad_rounds_2 N/A N/A testdir N/A N/A N/A test N/A N/A testdir makepyfile SIMPLE_TEST N/A result N/A N/A testdir runpytest --doctest-modules --benchmark-min-rounds=0 test N/A N/A N/A N/A result stderr fnmatch_lines N/A usage: py* [[]options[]] [[]file_or_dir[]] [[]file_or_dir[]] [[]...[]] py*: error: argument --benchmark-min-rounds: Value for --benchmark-rounds must be at least 1. N/A test_compare N/A N/A testdir N/A N/A N/A test N/A N/A testdir makepyfile SIMPLE_TEST N/A N/A N/A testdir runpytest --benchmark-max-time=0.0000001 --doctest-modules --benchmark-autosave test N/A result N/A N/A testdir runpytest --benchmark-max-time=0.0000001 --doctest-modules --benchmark-compare=0001 --benchmark-compare-fail=min:0.1 test N/A N/A N/A N/A result stderr fnmatch_lines N/A Comparing against benchmarks from: *0001_unversioned_*.json N/A result N/A N/A testdir runpytest --benchmark-max-time=0.0000001 --doctest-modules --benchmark-compare=0001 --benchmark-compare-fail=min:1% test N/A N/A N/A N/A result stderr fnmatch_lines N/A Comparing against benchmarks from: *0001_unversioned_*.json N/A test_compare_last N/A N/A testdir N/A N/A N/A test N/A N/A testdir makepyfile SIMPLE_TEST N/A N/A N/A testdir runpytest --benchmark-max-time=0.0000001 --doctest-modules --benchmark-autosave test N/A result N/A N/A testdir runpytest --benchmark-max-time=0.0000001 --doctest-modules --benchmark-compare --benchmark-compare-fail=min:0.1 test N/A N/A N/A N/A result stderr fnmatch_lines N/A Comparing against benchmarks from: *0001_unversioned_*.json N/A result N/A N/A testdir runpytest --benchmark-max-time=0.0000001 --doctest-modules --benchmark-compare --benchmark-compare-fail=min:1% test N/A N/A N/A N/A result stderr fnmatch_lines N/A Comparing against benchmarks from: *0001_unversioned_*.json N/A test_compare_non_existing N/A N/A testdir N/A N/A N/A test N/A N/A testdir makepyfile SIMPLE_TEST N/A N/A N/A testdir runpytest --benchmark-max-time=0.0000001 --doctest-modules --benchmark-autosave test N/A result N/A N/A testdir runpytest --benchmark-max-time=0.0000001 --doctest-modules --benchmark-compare=0002 -rw test N/A N/A N/A N/A result stdout fnmatch_lines N/A WBENCHMARK-C1 * Can't compare. No benchmark files * '0002'. N/A test_compare_non_existing_verbose N/A N/A testdir N/A N/A N/A test N/A N/A testdir makepyfile SIMPLE_TEST N/A N/A N/A testdir runpytest --benchmark-max-time=0.0000001 --doctest-modules --benchmark-autosave test N/A result N/A N/A testdir runpytest --benchmark-max-time=0.0000001 --doctest-modules --benchmark-compare=0002 test --benchmark-verbose N/A N/A N/A N/A result stderr fnmatch_lines N/A  WARNING: Can't compare. No benchmark files * '0002'. N/A test_compare_no_files N/A N/A testdir N/A N/A N/A test N/A N/A testdir makepyfile SIMPLE_TEST N/A result N/A N/A testdir runpytest --benchmark-max-time=0.0000001 --doctest-modules -rw test --benchmark-compare N/A N/A N/A N/A result stdout fnmatch_lines N/A WBENCHMARK-C2 * Can't compare. No benchmark files in '*'. Can't load the previous benchmark. N/A test_compare_no_files_verbose N/A N/A testdir N/A N/A N/A test N/A N/A testdir makepyfile SIMPLE_TEST N/A result N/A N/A testdir runpytest --benchmark-max-time=0.0000001 --doctest-modules test --benchmark-compare --benchmark-verbose N/A N/A N/A N/A result stderr fnmatch_lines N/A  WARNING: Can't compare. No benchmark files in '*'. Can't load the previous benchmark. N/A test_compare_no_files_match N/A N/A testdir N/A N/A N/A test N/A N/A testdir makepyfile SIMPLE_TEST N/A result N/A N/A testdir runpytest --benchmark-max-time=0.0000001 --doctest-modules -rw test --benchmark-compare=1 N/A N/A N/A N/A result stdout fnmatch_lines N/A WBENCHMARK-C1 * Can't compare. No benchmark files in '*' match '1'. N/A test_compare_no_files_match_verbose N/A N/A testdir N/A N/A N/A test N/A N/A testdir makepyfile SIMPLE_TEST N/A result N/A N/A testdir runpytest --benchmark-max-time=0.0000001 --doctest-modules test --benchmark-compare=1 --benchmark-verbose N/A N/A N/A N/A result stderr fnmatch_lines N/A  WARNING: Can't compare. No benchmark files in '*' match '1'. N/A test_verbose N/A N/A testdir N/A N/A N/A test N/A N/A testdir makepyfile SIMPLE_TEST N/A result N/A N/A testdir runpytest --benchmark-max-time=0.0000001 --doctest-modules --benchmark-verbose -vv test N/A N/A N/A N/A result stderr fnmatch_lines N/A   Timer precision: *s   Calibrating to target round *s; will estimate when reaching *s.     Measured * iterations: *s.   Running * rounds x * iterations ...   Ran for *s. N/A test_save N/A N/A testdir N/A N/A N/A test N/A N/A testdir makepyfile SIMPLE_TEST N/A result N/A N/A testdir runpytest --doctest-modules --benchmark-save=foobar --benchmark-max-time=0.0000001 test N/A N/A N/A N/A result stderr fnmatch_lines N/A Saved benchmark data in: * N/A N/A N/A json loads N/A N/A N/A N/A N/A N/A N/A N/A N/A N/A testdir tmpdir join .benchmarks listdir N/A 0 join 0001_foobar.json read N/A test_histogram N/A N/A testdir N/A N/A N/A test N/A N/A testdir makepyfile SIMPLE_TEST N/A result N/A N/A testdir runpytest --doctest-modules --benchmark-histogram=foobar --benchmark-max-time=0.0000001 test N/A N/A N/A N/A result stderr fnmatch_lines N/A Generated histogram: *foobar.svg N/A N/A N/A N/A f basename N/A f N/A N/A N/A testdir tmpdir listdir *.svg sort True N/A foobar.svg N/A test_autosave N/A N/A testdir N/A N/A N/A test N/A N/A testdir makepyfile SIMPLE_TEST N/A result N/A N/A testdir runpytest --doctest-modules --benchmark-autosave --benchmark-max-time=0.0000001 test N/A N/A N/A N/A result stderr fnmatch_lines N/A Saved benchmark data in: * N/A N/A N/A json loads N/A N/A N/A N/A N/A N/A N/A N/A N/A N/A N/A testdir tmpdir join .benchmarks listdir N/A 0 listdir 0001_*.json N/A 0 read N/A test_bogus_min_time N/A N/A testdir N/A N/A N/A test N/A N/A testdir makepyfile SIMPLE_TEST N/A result N/A N/A testdir runpytest --doctest-modules --benchmark-min-time=bogus test N/A N/A N/A N/A result stderr fnmatch_lines N/A usage: py* [[]options[]] [[]file_or_dir[]] [[]file_or_dir[]] [[]...[]] py*: error: argument --benchmark-min-time: Invalid decimal value 'bogus': InvalidOperation* N/A test_disable_gc N/A N/A testdir N/A N/A N/A test N/A N/A testdir makepyfile SIMPLE_TEST N/A result N/A N/A testdir runpytest --benchmark-disable-gc test N/A N/A N/A N/A result stdout fnmatch_lines N/A *collected 2 items test_disable_gc.py .. * benchmark: 2 tests * Name (time in ?s) * Min * Max * Mean * StdDev * Rounds * Iterations ------* test_fast          * test_slow          * ------* *====== 2 passed* seconds ======* N/A test_custom_timer N/A N/A testdir N/A N/A N/A test N/A N/A testdir makepyfile SIMPLE_TEST N/A result N/A N/A testdir runpytest --benchmark-timer=time.time test N/A N/A N/A N/A result stdout fnmatch_lines N/A *collected 2 items test_custom_timer.py .. * benchmark: 2 tests * Name (time in ?s) * Min * Max * Mean * StdDev * Rounds * Iterations ------* test_fast          * test_slow          * ------* *====== 2 passed* seconds ======* N/A test_bogus_timer N/A N/A testdir N/A N/A N/A test N/A N/A testdir makepyfile SIMPLE_TEST N/A result N/A N/A testdir runpytest --benchmark-timer=bogus test N/A N/A N/A N/A result stderr fnmatch_lines N/A usage: py* [[]options[]] [[]file_or_dir[]] [[]file_or_dir[]] [[]...[]] py*: error: argument --benchmark-timer: Value for --benchmark-timer must be in dotted form. Eg: 'module.attr'. N/A test_sort_by_mean N/A N/A testdir N/A N/A N/A test N/A N/A testdir makepyfile SIMPLE_TEST N/A result N/A N/A testdir runpytest --benchmark-sort=mean test N/A N/A N/A N/A result stdout fnmatch_lines N/A *collected 2 items test_sort_by_mean.py .. * benchmark: 2 tests * Name (time in ?s) * Min * Max * Mean * StdDev * Rounds * Iterations ------* test_fast          * test_slow          * ------* *====== 2 passed* seconds ======* N/A test_bogus_sort N/A N/A testdir N/A N/A N/A test N/A N/A testdir makepyfile SIMPLE_TEST N/A result N/A N/A testdir runpytest --benchmark-sort=bogus test N/A N/A N/A N/A result stderr fnmatch_lines N/A usage: py* [[]options[]] [[]file_or_dir[]] [[]file_or_dir[]] [[]...[]] py*: error: argument --benchmark-sort: Unacceptable value: 'bogus'. Value for --benchmark-sort must be one of: 'min', 'max', 'mean', 'stddev', 'name', 'fullname'. N/A test_xdist N/A N/A testdir N/A N/A N/A N/A N/A pytest importorskip xdist N/A test N/A N/A testdir makepyfile SIMPLE_TEST N/A result N/A N/A testdir runpytest --doctest-modules -n 1 -rw test N/A N/A N/A N/A result stdout fnmatch_lines N/A WBENCHMARK-U2 * Benchmarks are automatically disabled because xdist plugin is active.Benchmarks cannot be performed reliably in a parallelized environment. N/A test_xdist_verbose N/A N/A testdir N/A N/A N/A N/A N/A pytest importorskip xdist N/A test N/A N/A testdir makepyfile SIMPLE_TEST N/A result N/A N/A testdir runpytest --doctest-modules -n 1 --benchmark-verbose test N/A N/A N/A N/A result stderr fnmatch_lines N/A ------*  WARNING: Benchmarks are automatically disabled because xdist plugin is active.Benchmarks cannot be performed reliably in a parallelized environment. ------* N/A test_abort_broken N/A N/A testdir N/A N/A N/A 
    Test that we don't benchmark code that raises exceptions.
     N/A test N/A N/A testdir makepyfile 
"""
    >>> print('Yay, doctests!')
    Yay, doctests!
"""
import time
import pytest

def test_bad(benchmark):
    @benchmark
    def result():
        raise Exception()
    assert 1 == 1

def test_bad2(benchmark):
    @benchmark
    def result():
        time.sleep(0.1)
    assert 1 == 0

@pytest.fixture(params=['a', 'b', 'c'])
def bad_fixture(request):
    raise ImportError()

def test_ok(benchmark, bad_fixture):
    @benchmark
    def result():
        time.sleep(0.1)
    assert 1 == 0
 N/A result N/A N/A testdir runpytest -vv test N/A N/A N/A N/A result stdout fnmatch_lines N/A *collected 5 items test_abort_broken.py::test_bad FAILED test_abort_broken.py::test_bad2 FAILED test_abort_broken.py::test_ok[a] ERROR test_abort_broken.py::test_ok[b] ERROR test_abort_broken.py::test_ok[c] ERROR *====== ERRORS ======* *______ ERROR at setup of test_ok[[]a[]] ______* request = <SubRequest 'bad_fixture' for <Function 'test_ok[a]'>>     @pytest.fixture(params=['a', 'b', 'c'])     def bad_fixture(request): >       raise ImportError() E       ImportError test_abort_broken.py:22: ImportError *______ ERROR at setup of test_ok[[]b[]] ______* request = <SubRequest 'bad_fixture' for <Function 'test_ok[b]'>>     @pytest.fixture(params=['a', 'b', 'c'])     def bad_fixture(request): >       raise ImportError() E       ImportError test_abort_broken.py:22: ImportError *______ ERROR at setup of test_ok[[]c[]] ______* request = <SubRequest 'bad_fixture' for <Function 'test_ok[c]'>>     @pytest.fixture(params=['a', 'b', 'c'])     def bad_fixture(request): >       raise ImportError() E       ImportError test_abort_broken.py:22: ImportError *====== FAILURES ======* *______ test_bad ______* benchmark = <pytest_benchmark.*.BenchmarkFixture object at *>     def test_bad(benchmark): >       @benchmark         def result(): test_abort_broken.py:* _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _* * _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _*     @benchmark     def result(): >       raise Exception() E       Exception test_abort_broken.py:11: Exception *______ test_bad2 ______* benchmark = <pytest_benchmark.*.BenchmarkFixture object at *>     def test_bad2(benchmark):         @benchmark         def result():             time.sleep(0.1) >       assert 1 == 0 E       assert 1 == 0 test_abort_broken.py:18: AssertionError * benchmark: 1 tests * Name (time in ?s) * Min * Max * Mean * StdDev * Rounds * Iterations ------* test_bad2           * ------* *====== 2 failed*, 3 error* seconds ======* N/A N/A BASIC_TEST 
"""
Just to make sure the plugin doesn't choke on doctests::
    >>> print('Yay, doctests!')
    Yay, doctests!
"""
import time
from functools import partial

import pytest

def test_fast(benchmark):
    @benchmark
    def result():
        return time.sleep(0.000001)
    assert result is None

def test_slow(benchmark):
    assert benchmark(partial(time.sleep, 0.001)) is None

def test_slower(benchmark):
    benchmark(lambda: time.sleep(0.01))

@pytest.mark.benchmark(min_rounds=2)
def test_xfast(benchmark):
    benchmark(str)

def test_fast(benchmark):
    benchmark(int)
 test_basic N/A N/A testdir N/A N/A N/A test N/A N/A testdir makepyfile BASIC_TEST N/A result N/A N/A testdir runpytest -vv --doctest-modules test N/A N/A N/A N/A result stdout fnmatch_lines N/A *collected 5 items test_basic.py::*test_basic PASSED test_basic.py::test_slow PASSED test_basic.py::test_slower PASSED test_basic.py::test_xfast PASSED test_basic.py::test_fast PASSED * benchmark: 4 tests * Name (time in ?s) * Min * Max * Mean * StdDev * Rounds * Iterations ------* test_*         * test_*         * test_*         * test_*         * ------* *====== 5 passed* seconds ======* N/A test_skip N/A N/A testdir N/A N/A N/A test N/A N/A testdir makepyfile BASIC_TEST N/A result N/A N/A testdir runpytest -vv --doctest-modules --benchmark-skip test N/A N/A N/A N/A result stdout fnmatch_lines N/A *collected 5 items test_skip.py::*test_skip PASSED test_skip.py::test_slow SKIPPED test_skip.py::test_slower SKIPPED test_skip.py::test_xfast SKIPPED test_skip.py::test_fast SKIPPED *====== 1 passed, 4 skipped* seconds ======* N/A test_disable N/A N/A testdir N/A N/A N/A test N/A N/A testdir makepyfile BASIC_TEST N/A result N/A N/A testdir runpytest -vv --doctest-modules --benchmark-disable test N/A N/A N/A N/A result stdout fnmatch_lines N/A *collected 5 items test_disable.py::*test_disable PASSED test_disable.py::test_slow PASSED test_disable.py::test_slower PASSED test_disable.py::test_xfast PASSED test_disable.py::test_fast PASSED *====== 5 passed * seconds ======* N/A test_mark_selection N/A N/A testdir N/A N/A N/A test N/A N/A testdir makepyfile BASIC_TEST N/A result N/A N/A testdir runpytest -vv --doctest-modules -m benchmark test N/A N/A N/A N/A result stdout fnmatch_lines N/A *collected 5 items test_mark_selection.py::test_xfast PASSED * benchmark: 1 tests * Name (time in ?s) * Min * Max * Mean * StdDev * Rounds * Iterations ------* test_xfast       * ------* *====== 4 tests deselected by "-m 'benchmark'" ======* *====== 1 passed, 4 deselected* seconds ======* N/A test_only_benchmarks N/A N/A testdir N/A N/A N/A test N/A N/A testdir makepyfile BASIC_TEST N/A result N/A N/A testdir runpytest -vv --doctest-modules --benchmark-only test N/A N/A N/A N/A result stdout fnmatch_lines N/A *collected 5 items test_only_benchmarks.py::*test_only_benchmarks SKIPPED test_only_benchmarks.py::test_slow PASSED test_only_benchmarks.py::test_slower PASSED test_only_benchmarks.py::test_xfast PASSED test_only_benchmarks.py::test_fast PASSED * benchmark: 4 tests * Name (time in ?s) * Min * Max * Mean * StdDev * Rounds * Iterations ------* test_*         * test_*         * test_*         * test_*         * ------* *====== 4 passed, 1 skipped* seconds ======* N/A test_columns N/A N/A testdir N/A N/A N/A test N/A N/A testdir makepyfile SIMPLE_TEST N/A result N/A N/A testdir runpytest --doctest-modules --benchmark-columns=max,iterations,min test N/A N/A N/A N/A result stdout fnmatch_lines N/A *collected 3 items test_columns.py ... * benchmark: 2 tests * Name (time in ?s) * Max * Iterations * Min * ------* N/A