Package: pytest-passrunner
pytest-passrunner: A Pytest plugin that provides the ‘run_on_pass’ markerThepytest-passrunnerplugin for Pytest introduces a new marker calledrun_on_pass, which can be used to mark an entire test suite to only run a test if all previous tests in it passed successfully.
This allows for the user to use test structures, where previous tests setup future ones.
If previous tests were to fail, the future ones cannot run properly, and thus should be ‘xfailed’ (expected failure) instead of attempted anyway.
See below for some examples.Installation & UseHow to installpytest-passrunnercan be easily installed directly fromPyPIwith:$ pip install pytest-passrunnerIf required, one can also clone therepositoryand installpytest-passrunnermanually:$ git clone https://github.com/1313e/pytest-passrunner
$ cd pytest-success
$ pip install .Therun_on_passmarker can now be used in tests withpytest.mark.run_on_pass.Example useTherun_on_passmarker, when used on a Pytest suite, marks all tests in the suite to solely run if all previous tests in the suite passed.
Another way of saying this is that if a test in a marked test suite fails, all remaining tests are automatically flagged as ‘xfail’ (expected fail) and do not run.One use-case of this marker is when you want to write a test suite, where a specific object is being modified throughout the tests.
In this scenario, it is not uncommon that a particular test in the test suite can only run correctly if all previous tests passed, as otherwise the object is not in the expected state.
The following illustrates this scenario:# Import pytestimportpytest# Define Pytest suiteclassTest_list(object):# Create fixture that is shared between all tests in the suite# In this case, a list object@pytest.fixture(scope='class')deflst(self):# Create empty listlst=[]# Return itreturn(lst)# Perform test that sets up this listdeftest_1(self,lst):# Append 1 to the listlst.append(1)# Test that 1 was appendedassertlst==[1]# Perform test that requires lst to be [1]deftest_2(self,lst):# Append 2 to the listlst.append(2)# Test that 1 and 2 are in the list nowassertlst==[1,2]In this simple test script, a list object is being shared between all the tests in theTest_listtest suite.
In the second test,test_2, it is assumed that this list object was correctly set up by the first test.
Iftest_1were to fail for whatever reason,test_2cannot possibly pass anymore, as the list object was never set up properly.
However,test_2(and any further tests after it) will still be ran, wasting time and cluttering the final pytest summary overview with unhelpful errors.In order to solve this, we can apply therun_on_passmarker:# Import pytestimportpytest# Define Pytest suite with the 'run_on_pass' marker@pytest.mark.run_on_passclassTest_list(object):# Create fixture that is shared between all tests in the suite# In this case, a list object@pytest.fixture(scope='class')deflst(self):# Create empty listlst=[]# Return itreturn(lst)# Perform test that sets up this listdeftest_1(self,lst):# Append 1 to the listlst.append(1)# Test that 1 was appendedassertlst==[1]# Perform test that requires lst to be [1]deftest_2(self,lst):# Append 2 to the listlst.append(2)# Test that 1 and 2 are in the list nowassertlst==[1,2]If now the execution oftest_1fails,test_2(and any further tests after it in this test suite) will not be ran and be flagged as ‘xfail’ instead.
Tests that are xfailed do not generate any error messages in the pytest summary overview and are not seen as failed/errored tests either.
This keeps the summary cleaner, and no time is wasted on tests that cannot pass.