Package: zbs.zest
ZestA function-oriented testing framework for Python 3.Written by Zack Booth Simpson, 2020Available as a pip package:pip install zbs.zestMotivationPython's default unittest module is a class-oriented approach that
does not lend itself well to recursive setup and teardown.Zest uses a recursive function-based approach best demonstrated
with examples.########################################### some_module.pydef_say_hello():print("Hello")defunit_under_test(a):ifa<=0:raiseValueError("a should be positive")_say_hello()returna+1########################################### zest_some_module.pyfromzestimportzestimportsome_moduledefzest_unit_under_test():# This is a root-level zest because it starts with "zest_"defit_raises_on_non_positive():defit_raises_on_negative():withzest.raises(ValueError):some_module.unit_under_test(-1)defit_raises_on_zero():withzest.raises(ValueError):some_module.unit_under_test(0)zest()# Note this call which tells zest to run the above two testsdefit_calls_say_hello_once():withzest.mock(some_module._say_hello)asm_say_hello:some_module.unit_under_test(0)assertm_say_hello.called_once()zest()# Same here, this will cause it_raises_on_non_positive and it_calls_say_hello_once to runThe zest() function uses stack reflection to call each function that
it finds in the caller's stack-frame.  However, it only calls functions
that do not start with an underscore.Two special functions are reserved: _before() and _after()
which are called before/aftereachtest function in the scope.For example, often you may want to set up some complex state.defzest_my_test():state=Nonedef_before():nonlocalstatestate=State(1,2,3)defit_raises_if_bad():withzest.raises(Exception):unit_under_test(state)defit_modifies_state_on_1():unit_under_test(state,1)assertstate.foo==1defit_modifies_state_on_2():unit_under_test(state,2)assertstate.foo==2ExamplesSee./zests/zest_examples.pyfor more examples.UsageSearch recursively all directories for def zest_*() functions and execute them.$zestShow progress$zest--verbose=0# Show no progress$zest--verbose=1# Show "dot" progress (default)$zest--verbose=2# Show hierarchical full progressSearch only inside the specific dirs$zest--include_dirs=./abc:./defRun only tests that are in the "integration" or "slow" groups$zest--groups=integration:slowRun only tests that contain the string "foobar". This will also
run any parent test needed to execute the match.$zestfoobarDisable test order shuffling which is on by default to increase the
liklihood that accidental order-dependencies are manifest.$zest--disable_shuffleHelpersExpected exceptionsdefzest_foobar_should_raise_on_no_arguments():withzest.raises(ValueError):foobar()Sometimes you may wish to check a property of the trapped exceptiondefzest_foobar_should_raise_on_no_arguments():withzest.raises(ValueError)ase:foobar()asserte.exception.args==("bad juju",)Often you may wish to check only for a string of a property of the trapped exception
in which case you can use the in_* argument to the raises.defzest_foobar_should_raise_on_no_arguments():withzest.raises(ValueError,in_args="bad juju")ase:foobar()Mocksimportunit_under_testdefzest_foobar():withzest.mock(unit_under_test.bar)asm_bar:# Suppose unit_under_test.foobar() calls bar()m_bar.returns(0)unit_under_test.foobar()assertm_bar.called_once_with(0)Seezest.MockFunctionfor a complete MockFunction API.GotchasDon't forget to put the zest() call at each level of the test.
If you forget, the zest runner will throw an error along the lines of:
"function did not terminate with a call to zest()..."defzest_something():defit_foos():foo()defit_bars():bar()# WRONG! zest() wasn't called here. Error will be thrown when the test is run.Do not mock outside of test functions:defzest_something():withzest.mock(...):defit_does_something():assertsomethingdefit_does_something_else():assertsomething# The zest() will execute outside of the above "with" statement so# the two tests will not inherit the mock as expected.zest()Rather, put the zest() inside the "with mock":defzest_something():withzest.mock(...):defit_does_something():assertsomethingdefit_does_something_else():assertsomething# This is fine because zest() was called INSIDE the withzest()Don't have more than one zest() call in the same scope.defzest_something():withzest.mock(...):defit_does_something():assertsomethingdefit_does_something_else():assertsomething# Like above example; so far, so good, but watch out...zest()withzest.mock(...):defit_does_yet_another_thing():assertsomething# WRONG! A second call to zest() will RE-EXECUTE the above two tests# (it_does_something and it_does_something_else) because this# second call to zest() doesn't know that it is inside of a with statement.# The "with" scope makes it look different but really the following# call to zest() and the call to zest above are actually in the same scope.zest()When asserting on properties of an expected exception,
be sure to do assert outside the scope of the "with" as demonstrated:Wrong:withzest.raises(SomeException)ase:something_that_raises()asserte.exception.property=="something"# The above "assert" will NOT be run because the exception thrown by# something_that_raises() will be caught and never get to execute the assert!Right:withzest.raises(SomeException)ase:something_that_raises()asserte.exception.property=="something"# (Note the reference to "e.exception." as opposed to "e."Remember that the exception returned from a zest.raises() isnotof the type you are expecting but rather of a wrapper
class calledTrappedException. To get to the properties
of interest you need to usee.exception.*.Wrong:withzest.raises(SomeException)ase:something_that_raises()asserte.property=="something"# Wrong! e is of type TrappedException therefore the above will not work as expected.Right:withzest.raises(SomeException)ase:something_that_raises()asserte.exception.property=="something"# Correct, .exception reference to get original exception from the `e` TrappedException wrapper.DevelopmentRun in development modepipenvshell
pipenvsync# Run all the example tests (which actually test the tester itself).$./zest.shDeploy$./deploy.shYou will need the user and password and credentials for Pypi.orgTODOConvert over to using logging correctlyWhen debug mode is on in ui, and a test runs, you don't see the success increment--ui fails is brokenWhen match string matches nothing it is confusing. Need "nothing was run"Add a "slowest last" to UIAdd "raises" to mock and stack mock. And error if BOTh returns and raises are setAdd --rng_seed optionMake searches more clear -- currently hard-coded to only search "zests" directoriesHarden failed imports on zest runner AST importMirror debugger-like stack introspection into a set of check-like helpers for type, arrays, etc.Add a zest internal test that _after is called even if the subsequent test exceptions (that is, _after is in a finally block)Add coverage