Package: pytest-coveragemarkers
Using pytest markers to track functional coverage and filtering of testsThispytestplugin was generated withCookiecutteralong with@hackebrot’scookiecutter-pytest-plugintemplate.FeaturesDefinition of CoverageMarkers© in YAML formatSupport for applying CoverageMarkers© to testsFiltering of tests based on CoverageMarkers©Inclusion of CoverageMarkers© in JSON reportInstallationYou can install “pytest-coveragemarkers” fromPyPI:$ pip install pytest-coveragemarkers
# or
$ poetry add pytest-coveragemarkersUsageStep 1: Define your coverage markers yaml.Using the format:markers:-name:<marker_name>allowed:-<marker_value_1>-<marker_value_2>-name:<marker2_name>allowed:-<marker2_value_1>-<marker2_value_2>Then decorate your tests with themimportpytest@pytest.mark.marker_name(['value1','value2'])@pytest.mark.marker2_name(['value1','value2'])deftest_x():...@pytest.mark.marker2_name(['value1','value2'])deftest_y():...Then when the tests are executed withpytest--json-report--markers-location=/full/path/to/coverage_markers.ymlThen the JSON Test Report output from the test execution contains:"tests":[{"nodeid":"...","metadata":{"cov_markers":{"marker_name":{"value1":true,"value2":true},"marker2_name":{"value1":true,"value2":true}}}},...]This can then be used to generate test coverage details based on the coverage markers.
A nice demo will be produced to give examples of usage.But wait there is another benefit:We can filter tests for execution based on their coverage markerspytest\--filter='{"and": [{"eq": ["marker_name.value1", true]}]}'\--json-report\--markers-location=/full/path/to/coverage_markers.ymlThe above command run against the tests defined above would select ‘test_x’ and deselect ‘test_y’ for executionOther examples of filters are:You can also supply the path to a json file containing your filter.
Use argument –filter-location or key FilterLocation in the pytest.ini file.Coverage Marker Argument FormatThe arguments supplied to Coverage Markers can follow multiple formats which allows the user to define the format that best suites them.E.g.importpytest@pytest.mark.marker_1('value1')# single string argument@pytest.mark.marker_2('value1','value2')# multiple string arguments@pytest.mark.marker_3(['value1','value2'])# list of arguments@pytest.mark.marker_4(('value1','value2'))# tuple of argumentsdeftest_x():...TestingNox is used by this project to execute all tests.
To run a specific set of tests execute the below line:$ poetry run nox -s <session_name>Where session_name can be one of the followingNox SessionsSession NameSession Detailsunit_testsExecute all tests marked as unitfunctional_testsExecute all tests marked as functionalThought ProcessThepytest_docstalks about using markers to set metadata on tests and use the markers to select required tests for execution.For the markers I want to add, I also want to specify a list of values that go along with that marker.
E.g. If the marker was ‘colour’ then supported values may be ‘Red’, ‘Green’, ‘Gold’.I also want the list of values validated against supported values so no unsupported values can be added.
E.g. If the marker was ‘colour’ then a value of ‘Panda’ would not be allowed.Then all this meta data I want to come out in the junit json report.Next I want to use these markers and their supported values to filter tests. For this I need a more powerful filter engine.DocumentationTo build the docs run:poetry run mkdocs serveLicenseDistributed under the terms of theMITlicense, “pytest-coveragemarkers” is free and open source softwareIssuesIf you encounter any problems, pleasefile an issuealong with a detailed description.Future ChangesType-HintsFull Test CoverageFull DocumentationRefactor/rewritePick one, yml or json?