Package: repeated-test
repeated_testlets you write tests that apply the same function to
many sets of parameters.For instance:fromrepeated_testimportFixturesclassMyFixtures(Fixtures):def_test(self,expected,*terms):self.assertEqual(expected,sum(terms))a=10,5,5b=15,7,8c=42,1,1The result is unittest-compatible, and provides useful context in the
traceback in case of errors:$python-munittestmy_tests..F
======================================================================
FAIL: test_c (my_tests.MyFixtures)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "my_tests.py", line 9, in MyFixtures
    c = 42, 1, 1
  File "my_tests.py", line 5, in _test
    self.assertEqual(expected, sum(terms))
AssertionError: 42 != 2

----------------------------------------------------------------------
Ran 3 tests in 0.002s

FAILED (failures=1)You can install it using:$pipinstall--userrepeated_testHelp / IssuesYou can get help in thegitter.im chatroom.If you find any issues or have any requests, useGitHub Issues.ReferenceIntroductionPython’sunittestmodules helps in performing various forms of automated testing.
One writes a class deriving fromunittest.TestCaseand adds varioustest_xyzmethods.
Test runners run these tests, keeping count of successful and failed tests,
and produces a trace with the causes of these failures.Sometimes it makes sense to have one test be carried out for a large amount
of different inputs.
This module aims to provide an efficient way to do this.It allows you to write fixtures (inputs) as plain members of a
class, and bind a test function to them. This test function is called for each
fixture as you will see below. The produced class is aunittest.TestCasesubclass, so it is
compatible withunittestand other unittest-compatible test runners.Building a test caseIn order to produce aunittest.TestCase,repeated_testrequires you to:Subclassrepeated_test.FixturesWrite a_testmethod that takes a few parameters, making use of anyunittest.TestCasemethod it needsAssign fixtures directly in the class body, which are then unpacked as
arguments to the_testmethod (as in_test(*args))You can use anyunittest.TestCasemethods in your test function, such asassertEqual()and so forth.fromrepeated_testimportFixturesclassMyFixtures(Fixtures):def_test(self,arg1,arg2,arg3):self.assertEqual(...,...)Ps='p1','p2','p3'# _test(*Ps) will be called, ie. _test('p1', p2', 'p3')Qs='q1','q2','q3'# _test(*Qs) will be called, ie. _test('q1', q2', 'q3')Make sure that your fixture tuples provide the correct amount of arguments
for your_testmethod, unless it has an*argsparameter.Running a test caseYou can run arepeated_testtest case like any otherunittest.TestCaseclass:python-munittestpython-munittestmy_test_modulepython-munittestmy_test_module.MyFixtures# To refer to an individual test, prefix the name of the fixture with "test_"python-munittestmy_test_module.MyFixtures.test_PsLearn more in theofficial unittest docs.You can also use a unittest-compatible test runer, likepytest:python-mpytestpython-mpytestmy_test_module.pypython-mpytestmy_test_module.py-kMyFixturespython-mpytestmy_test_module.py-ktest_Pspython-mpytestmy_test_module.py::MyFixtures::test_PsLearn more in theofficial pytest docsPassing in keyword argumentsYou can pass in keyword arguments usingrepeated_test.options:importsysfromrepeated_testimportFixtures,optionsclassMyFixtures(Fixtures):def_test(self,arg1,arg2,*,min_version=None,max_version=None):...not_using_versions="abc","abc"# -> _test("abc", "abc")using_max_version="abc","abc",options(max_version=(3,9))# -> _test("abc", "abc", max_version=(3, 9))using_both_versions="abc","abc",options(min_version=(3,8),max_version=(3,9))# -> _test("abc", "abc", min_version=(3, 8), max_version=(3, 9))using_both_versions_2="abc","abc",options(min_version=(3,8)),options(max_version=(3,9))# Same, but by specifying options separatelyThis can be useful if you have multiple options that are only used some of the time.Passing in keyword arguments to multiple testsIf you are re-using the same keyword arguments across multiple tests,
there are several ways to do so:Using@repeated_test.with_options(...)lets you
specify options for every fixture within a class.Usingwithrepeated_test.options(...)lets you
specify options for every fixture within thewithblock.You can continue usingoptions()on individual fixtures,
and override values provided by surrounding code.You can userepeated_test.skip_optionto stop supplying any argument.
The argument won’t be supplied to the function,
which is useful for using a default value,
or for removing common options from a class whose test function doesn’t handlefromrepeated_testimportFixtures,options,with_options@with_options(kwarg1="value from decorator")classMyFixtures(Fixtures):def_test(self,arg1,arg2,*,kwarg1,kwarg2="default"):...using_provided_values="arg1","arg2"# -> _test("arg1", "arg2", kwarg1="value from decorator", kwarg2="default")overriding_provided_values="arg1","arg2",options(kwarg1="kwarg1",kwarg2="kwarg2")# -> _test("arg1", "arg2", kwarg1="kwarg1", kwarg2="kwarg2")withoptions(kwarg1="value from context manager"):using_value_from_context_manager="arg1","arg2"# -> _test("arg1", "arg2", kwarg1="value from context manager", kwarg2="default")overriding_value_from_context_manager="arg1","arg2",options(kwarg1="kwarg1")# -> _test("arg1", "arg2", kwarg1="kwarg1", kwarg2="default")withoptions(kwarg2="value from context manager"):removing_value="arg1","arg2"# -> _test("arg1", "arg2", kwarg1="value from decorator", kwarg2="value from context manager")removing_value="arg1","arg2",options(kwarg2=skip_option)# -> _test("arg1", "arg2", kwarg1="value from decorator", kwarg2="default")Testing multiple values for a keyword parameterYou can also use@with_options_matrixto provide multiple values for a keyword parameter.repeated_testwill run every combination
except for parameters that are overridden.fromrepeated_testimportFixtures,options,with_options_matrix@with_options_matrix(spam=["spam1","spam2"],ham=["ham1","ham2"],)classMyFixtures(Fixtures):def_test(self,arg1,arg2,*,spam,ham):...using_provided_values="arg1","arg2"# -> _test("arg1", "arg2", spam="spam1", ham="ham1")# -> _test("arg1", "arg2", spam="spam1", ham="ham2")# -> _test("arg1", "arg2", spam="spam2", ham="ham1")# -> _test("arg1", "arg2", spam="spam2", ham="ham2")withoptions(spam="spam"):overriding_one_value="arg1","arg2"# -> _test("arg1", "arg2", spam="spam", ham="ham1")# -> _test("arg1", "arg2", spam="spam", ham="ham2")repeated_testwill report each combination using unittest’ssubtestsfeature.
pytest does not have this feature built-in, but thepytest-subtestsplugin adds support.======================================================================
FAIL: test_overriding_one_value (example_options._test) (ham='ham1')
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/myself/repeated_test/example_options.py", line 41, in MyFixtures
    overriding_one_value = "arg1", "arg2"
  File "/home/myself/repeated_test/example_options.py", line 32, in _test
    self.fail("example failure")
AssertionError: example failure

======================================================================
FAIL: test_overriding_one_value (example_options._test) (ham='ham2')
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/myself/repeated_test/example_options.py", line 41, in MyFixtures
    overriding_one_value = "arg1", "arg2"
  File "/home/myself/repeated_test/example_options.py", line 32, in _test
    self.fail("example failure")
AssertionError: example failureEvaluated test case inputYou can use@evaluatedto make
a function that can be inserted in a test case tuple.
When running the test,
the function will be called,
and the result will be spliced into the test case tuple:fromrepeated_testimportFixtures,evaluatedclassEvaluatedFixtures(Fixtures):def_test(self,a,b,c):pass@evaluateddeffully_evaluated(self):return(1+1,2,3)# -> _test(2, 2, 3)@evaluateddef_helper():return(1+2,3)partly_evaluated=_helper(),4# -> _test(3, 3, 4)The wrapped function must always return a tuple.The wrapped function will also receive options as keyword arguments:fromrepeated_testimportFixtures,evaluated,with_options_matrix@with_options_matrix(option=["option 1","option 2",],)classEvaluatedFixturesWithOptions(Fixtures):def_test(self,a,*,option):pass@evaluateddef_helper(arg,*,option):return(f"{arg}-{option}",)using_option_in_evaluated=_helper("arg")# -> _test("arg-option 1", option="option 1")# -> _test("arg-option 2", option="option 2")Named alternativesWhen usingwith_options_matrix,
sometimes the repr generated by those options is difficult to read.
This can be the case with functions,
which can end up showing as something like<function myfunc at 0x7f9f5e506280>.You can useNamedAlternativeto give them a name:fromrepeated_testimportNamedAlternativevalue1=NamedAlternative("name for value1","value1")@NamedAlternative("name for func1")deffunc1():passNaming and escapingYou may name your test tuples however you like, though they may not start withtest_or_. They are copied to the resultingunittest.TestCaseclass, and test
methods are created for them. Their name is that of the tuple, prefixed withtest_.Members starting withtest_or_are directly copied over to the
resultingunittest.TestCaseclass, without being treated as fixtures. You can use this to
insert regular tests amongst your fixtures, or constants that you do not wish
to be treated as tests:fromrepeated_testimportFixturesclassMyFixtures(Fixtures):def_test(self,arg1,arg2,arg3):self.assertEqual(...,...)deftest_other(self):self.assertEqual(3,1+2)_spam='spam, bacon and eggs'# _spam won't be treated as a fixture, so _test(*_spam) won't be calledham=_spam,_spam,_spamYou may even call the test function usingself._test(...)if necessary.Separating tests and fixturesYou can apply a fixtures class to a different test function using itswith_testmethod:classMyFixtures(Fixtures):_test=None...@MyFixtures.with_testdefother_test(self,arg1,arg2,arg3):self.assertEqual(...,...)While the function appears out of any class, it will be used as a method of
the resultingunittest.TestCaseclass, so keep in mind that it takes aselfparameter.You can reuse a fixture class however many times you like.If you specify a test function this way, you can set_test = Nonein your fixtures definition. However, it will not be discovered byunittest,
soregular test methodswon’t be run.
Omitting_testcompletely raises an error in order to prevent accidentally
disabling your tests.Working with functions as fixturesIt can be fairly impractical to use functions in your fixture tuples in this
scheme. If your fixture tuple is meant to have one function in it, you can
use thetupdecorator:fromrepeated_testimportFixtures,tupclassmy_tests(Fixtures):def_test(self,func,arg1,arg2):self.assertEqual(...,...)@tup('arg1','arg2')defham():pass# equivalent todef_ham():passham=_ham,'arg1','arg2'Replacing unittest.TestCase with another classYou can replace unittest.TestCase with another class usingWithTestClass(cls).For instance, if you wish to useunittest2:importunittest2fromrepeated_testimportWithTestClassclassmy_tests(WithTestClass(unittest2.TestCase)):...