Package: pytest-idempotent
pytest-idempotentPytest plugin for testing the idempotency of a function.Usagepip install pytest-idempotentDocumentationSuppose we had the following function, that we (incorrectly) assumed was idempotent. How would we write a test for this?First, we can label the function with a decorator:# abc.pyfrompytest_idempotentimportidempotent# or use your own decorator! See below.@idempotentdeffunc(x:list[int])->None:x+=[9]Note: this function isnotidempotent because calling it on the same listxgrows the size ofxby 1 each time. To be idempotent, we should be able to runfuncmore than once without any adverse effects.We can write an idempotency test for this function as follows:# tests/abc_test.pyimportpytest@pytest.mark.idempotentdeftest_func()->None:x:list[int]=[]func(x)assertx==[9]Adding the@pytest.mark.idempotentmark automatically splits this test into two - one that tests the regular behavior, and one that tests that the function can be called twice without adverse effects.❯❯❯ pytest

================= test session starts ==================
platform darwin -- Python 3.9.2, pytest-6.2.5
collected 2 items

tests/abc_test.py .F                     [100%]

=====================  FAILURES ========================
------------- test_func[idempotency-check] -------------

    @pytest.mark.idempotent
    def test_func() -> None:
        x: list[int] = []

        func(x)

>       assert x == [9]
E       assert [9, 9] == [9]
E         Left contains one more item: 9
E         Use -v to get the full diff

tests/abc_test.py:19: AssertionError
=============== short test summary info ================
FAILED tests/abc_test.py::test_func[idempotency-check]
  - assert [9, 9] == [9]
============= 1 failed, 1 passed in 0.16s ==============How It WorksIdempotency is a difficult pattern to enforce. To solve this issue,pytest-idempotenttakes the following approach:Introduce a decorator,@idempotent, to functions.This decorator serves as a visual aid. If this decorator is commonly used in the codebase, it is much easier to consider idempotency for new and existing functions.At runtime, this decorator is a no-op.At test-time, if the feature is enabled, we will run the decorated function twice with the same parameters in all test cases.We can also assert that the second run returns the same result using an additional parameter to the function's decorator:@idempotent(equal_return=True).For all tests marked using@pytest.mark.idempotent, we run each test twice: once normally, and once with the decorated function called twice.Both runs need to pass all assertions.We return the first result because the first run will complete the processing. The second will either return exact the same result or be a no-op.To disable idempotency testing for a test or group of tests, add the Pytest marker:@pytest.mark.idempotent(enabled=False)Enforcing Tests Use@pytest.mark.idempotentBy default, any test that calls an@idempotentfunction must also be decorated with the marker@pytest.mark.idempotent.To disable idempotency testing for a test or group of tests, use:@pytest.mark.idempotent(enabled=False), or add the following config to your project:defpytest_idempotent_enforce_tests()->bool:returnFalseTo disable enforced idempotency testing for a specific function, you can also pass the flag into the decorator:# abc.pyfrompytest_idempotentimportidempotent@idempotent(enforce_tests=False)deffunc()->None:returnOr, you can automatically add the marker based on the test name by adding toconftest.py:# conftest.pydefpytest_collection_modifyitems(items):foriteminitems:if"idempotent"initem.nodeid:item.add_marker(pytest.mark.idempotent)@idempotent decoratorBy default, the@idempotentdecorator does nothing during runtime. We do not want to add overhead to production code to run tests.fromtypingimportAny,Callable,TypeVar_F=TypeVar("_F",bound=Callable[...,Any])defidempotent(func:_F)->_F:"""No-op during runtime.This marker allows pytest-idempotent to override the decorated functionduring test-time to verify the function is idempotent."""returnfuncTo use your own@idempotentdecorator, you can override thepytest_idempotent_decoratorfunction in yourconftest.pyto return the module path to your implementation.# conftest.py# Optional: you can define this to ensure the plugin is correctly installedpytest_plugins=["pytest_idempotent"]defpytest_idempotent_decorator()->str:# This links to my custom implementation of @idempotent.return"src.utils.idempotent"