Package: testing-fixtures
Beyond Pytest FixturesThis repo contains an implementation of a new approach to fixtures for use withpytest.
In addition we demonstrate how to use these new fixtures in both unit and
integration tests.Utility FixturesThe library also comes with some utility fixtures that often come in handy.
For examplecreate_temp_dir(injects thePathto a temporary directory into the test using the fixture) andcreate_temp_cwd(switches the cwd to a temporary directory and
injects itsPathinto the test).Project EvolutionThe evolution of this project is being tracked in thisdoc.Advantages ofpytestfixturespytestfixtures are extremely powerful and flexible.Provides a pythonic mechanism for setup, teardown, and injection of state into tests.Fixture scope is configurable allowing for heavy computation to be
carried out once per test function (default), test module or session.Allows fixtures to be composed which sets up a causal relation between fixtures and
sharing of state between multiple fixtures and the test function.Disadvantages ofpytestfixturesTunabilitypytestfixtures lack a straight-forward mechanism for passing arguments to them from
the test function defition site.
It is not uncommon to require that a specific piece of state be injected before
running a specific test.
A "tunable" fixture would solve this requirement.pytestsolves this bymagicallyallowingpytest.mark.parametrizevalues to be
passed through to a fixture being used by a test.
This is not obvious and uses a mechanism that is primarily used for
injecting multiple states into a test to create multiplicity.Importabilitypytestrecommends that fixtures be defined in aconftest.pyfile (a most non-obvious
name) and that theynotbe imported directly.
When tests are executedpytestparsesconftest.pyand magically inserts
the fixtures (setup, teardown, and interjection) into the test execution.
This is completely different from how the rest of Python operates and
is a source of great confusion to newcomers.Fixtures vs Injected Valuepytestfixtures overlap two distinct concepts when connecting a test function to
a fixture.
One is the name/handle to the fixture definition (generator function), and
the other is the variable inside the test function which is
bound to the value yielded by the fixture.This over-use of a single name is evident every time one is choosing the name for
the fixture + variable.
Does one name it for the variable or for the operation that the fixture carries out
whose side-effect is the value in the varible e.g.add_portfoliovsportfolio_name.Type AnnotationThe waypytestregistersfixtures and theninjects/interleavesthem into/with
test functions means it is practically impossible for a type engine to match and
enforce types between the fixture definition and the value injected into
the test function.This is a source of considerable frustration for anyone who has gone through the
effort to annotate their code and their tests.PrototypeWe provide a prototype for a new type of fixtures beyond what is provided bypytest.ObjectivesWorks seamlessly withpytest.Importable from another module (no moreconftest.py).Composable.
One fixture can be connected to another fixture and recieve a value from it.Tunable.
Fixtures definitions can declare parameters.
These parameters can either be provided ateitherthe test definition siteorinside the fixture definition module.The value(s) provided to the parameter(s) will remain consistent throughout
the execution of any given test.
The same value will be visible to all participating entities:
the test function and all fixtures composed with said fixture.Fully typed and type-aware.
Provides enforceable bindings between fixture definitions,
values injected into fixtures, and
values injected from them into test functions.InterfaceTo achieveallof the objectives listed above the interface for these fixtures is
slightly more verbose thanpytestfixtures while
being significantly less magical.The following four decorators are provided for defining these fixtures:@fixture: Applied to a fixture definition (one-shot generator function).
Creates an instance of theFixtureclass.This instance is both a decorator as well as a
reusable and reentrant context manager.This instance is applied as a decorator to test functions and injects
the yielded value into it.Example (extracted fromtests/unit/utils.py):@fixturedeffixture_b(b1:Bi1,b2:Bi2)->FixtureDefinition[Bo]:"""A fixture that takes injected value from the test function decoration."""yieldBo(b1=b1,b2=b2)NoteOne can useNewTypeandTypedDictto constrain the fixture parameters and
the value it yields which allows for tightly binding the yielded value
to any location where it is used (test function or composed fixture).
Similarly theFixtureDefinition[]generic type constrains how the fixture is
allowed to be used.Each instance has a.set()method which is used to provide values for
any parameters declared in the fixture definition..set()can be called on either the test function decoration site,
inside the module defining the fixture,
or while composing the fixture with another.Examples (extracted fromtests/unit/utils.pyandtests/unit/test_new_fixtures.py):@fixture_b.set(Bi1(42),Bi2(3.14))deftest_b(b:Bo)->None:"""Test parametrized fixture_b in isolation."""assertb=={"b1":42,"b2":3.14}or@fixture@compose(fixture_b.set(Bi1(13),Bi2(1.44)))deffixture_c(b:Bo)->FixtureDefinition[Co]:"""A fixture that takes an injected value from ANOTHER fixture."""yieldCo(c=b)All fixture composition and test decoration creates a strict ordering of when the
fixture's context manager is entered and exited.
Only the value at the first entry is available throughout the execution of a single
test.@compose: A function that takes a single argument which must be an instance ofFixtureand returns a decorator that is applied to another fixture definition.
Designed to be appliedbeforethe fixture definition is wrapped inside@fixture(don't worry,
the type system will shout at you if you get the order wrong).The value yielded by the composed fixture is injected as the first parameter of
the decorated fixture definition.
It essentially creates a simpler fixture definition with one less parameter.Example:@fixture@compose(fixture_b.set(Bi1(13),Bi2(1.44)))deffixture_c(b:Bo)->FixtureDefinition[Co]:"""A fixture that takes an injected value from ANOTHER fixture."""yieldCo(c=b)The composed fixture can also have its value set from the test site but
be available to the composition.Example:@fixture@compose(fixture_b)deffixture_g(b:Bo,g:Gi)->FixtureDefinition[Go]:"""Fixture that uses a late-injected fixture_b and a value from the test site."""yield{"b":b,"g":g}@fixture_b.set(Bi1(56),Bi2(9.7))@fixture_g.set(Gi(41))deftest_g(g:Go,b:Bo)->None:"""Inject args into fixture from test site and trickle down to pulled in fixture."""assertb=={"b1":56,"b2":9.7}assertg=={"b":b,"g":41}@noinject: Used at the test definition decoration site to wrap fixtures.
The wrapped fixture's yielded values willnotbe injected into the test function.Example:@noinject(fixture_b.set(Bi1(75),Bi2(2.71)))deftest_b_no_injection()->None:"""The value yielded by fixture_b is NOT injected into the test."""The type engine will understandnotinjecting and validates accordingly.@compose_noinject: Applied to composed fixtures tostopthem from injecting
their yielded value into the fixture they are composed with.Example:@fixture@compose_noinject(fixture_b.set(Bi1(39),Bi2(8.1)))deffixture_h(h:Hi)->FixtureDefinition[Ho]:"""Fixture that uses a composed fixture_b but NOT its yielded value."""yieldHo(h=h)Again, the type engine is aware of the mechanics.ImplementationThe implementation can be found intesting.fixtures.
It consists of nested decorators, modified context managers, and parameter injection,
all fully typed.DemoIntegration TestsTo demonstrate this in action we have a REST server that:receives POST requestsfetches data from a (postgres) databaseuses the fetched data to construct the responseThis has been setup as a composable environment.
First build the local images:docker-compose build.
Then, run the tests:docker-compose run --rm test.Unit TestsWe have also provided a number of unit tests, unrelated to the REST server application,
which focus on demonstrating all the possible permutations of fixture usage,
composition, and state injection at both the test and fixture definition site.Local DevelopmentTo make changes to this code base the recommendation is to use a virtual env:python3.11 -m venv .venvsource .venv/bin/activatepip install ".[dev]"Your IDE should be able to now access this virtual env and
provide you with autocomplete, intellisense, etc.How to DeployBuild the package:python3.11 -m buildThis will create the source tarball and wheel in thedist/folder.Deploy to pypi:python3.11 -m twine upload dist/*Enter your pypi username and password.