Package: zope.testrunner
zope.testrunnerContentszope.testrunnerUsing zope.testrunnerInstallationBuildout-based projectsVirtualenv-based projectsSome useful command-line options to get you startedWriting testsTest groupingOther featureszope.testrunner Changelog6.4 (2024-02-27)6.3.1 (2024-02-12)6.3 (2024-02-07)6.2.1 (2023-12-22)6.2 (2023-11-08)6.1 (2023-08-26)6.0 (2023-03-28)5.6 (2022-12-09)5.5.1 (2022-09-07)5.5 (2022-06-24)5.4.0 (2021-11-19)5.3.0 (2021-03-17)5.2 (2020-06-29)5.1 (2019-10-19)5.0 (2019-03-19)4.9.2 (2018-11-24)4.9.1 (2018-11-21)4.9 (2018-10-05)4.8.1 (2017-11-12)4.8.0 (2017-11-10)4.7.0 (2017-05-30)4.6.0 (2016-12-28)4.5.1 (2016-06-20)4.5.0 (2016-05-02)4.4.10 (2015-11-10)4.4.9 (2015-05-21)4.4.8 (2015-05-01)4.4.7 (2015-04-02)4.4.6 (2015-01-21)4.4.5 (2015-01-06)4.4.4 (2014-12-27)4.4.3 (2014-03-19)4.4.2 (2014-02-22)4.4.1 (2013-07-10)4.4.0 (2013-06-06)4.3.3 (2013-03-03)4.3.2 (2013-03-03)4.3.1 (2013-03-02)4.3.0 (2013-03-02)4.2.0 (2013-02-12)4.1.1 (2013-02-08)4.1.0 (2013-02-07)4.0.4 (2011-10-25)4.0.3 (2011-03-17)4.0.2 (2011-03-16)4.0.1 (2011-02-21)4.0.0 (2010-10-19)4.0.0b5 (2010-07-20)4.0.0b4 (2010-06-23)4.0.0b3 (2010-06-16)4.0.0b2 (2010-05-03)4.0.0b1 (2010-04-29)This package provides a flexible test runner with layer support.Detailed documentation is hosted athttps://zopetestrunner.readthedocs.ioUsing zope.testrunnerInstallationBuildout-based projectszope.testrunner is often used for projects that usebuildout:[buildout]
develop = .
parts = ... test ...

[test]
recipe = zc.recipe.testrunner
eggs = mypackageThe usual buildout processpython bootstrap.py
bin/buildoutcreates abin/testscript that will run the tests formypackage.Tipzc.recipe.testrunnertakes care to specify the right--test-pathoption in the generated script.  You can add
other options (such as--tests-pattern) too; checkzc.recipe.testrunner’s documentation for details.Virtualenv-based projectspip install zope.testrunnerand you’ll get azope-testrunnerscript.  Run your tests withzope-testrunner --test-path=path/to/your/source/treeYour source code needs to be available for the testrunner to import,
so you need to runpython setup.py installorpip install-e.into the samevirtualenv.Some useful command-line options to get you started-pshow a progress indicator-vincrease verbosity-ccolorize the output-ttestspecify test names (one or more regexes)-mmodulespecify test modules (one or more regexes)-spackagespecify test packages (one or more regexes)--list-testsshow names of tests instead of running them-xstop on first error or failure-D,--pdbenable post-mortem debugging of test failures--xmlpathgenerate XML reports to be written at the given path--helpshowallcommand-line options (there are many more!)For examplebin/test -pvc -m test_foo -t TestBarruns all TestBar tests from a module called test_foo.py.Writing testszope.testrunnerexpects to find your tests inside your package
directory, in a subpackage or module namedtests.  Test modules
in a test subpackage should be namedtest*.py.TipYou can change these assumptions with--tests-patternand--test-file-patterntest runner options.Tests themselves should be classes inheriting fromunittest.TestCase, and if you wish to use doctests, please tell
the test runner where to find them and what options to use for them
in by supplying a function namedtest_suite.Example:import unittest
import doctest

class TestArithmetic(unittest.TestCase):

    def test_two_plus_two(self):
        self.assertEqual(2 + 2, 4)


def doctest_string_formatting():
    """Test Python string formatting

        >>> print('{} + {}'.format(2, 2))
        2 + 2

    """

def test_suite():
    return unittest.TestSuite([
        unittest.defaultTestLoader.loadTestsFromName(__name__),
        doctest.DocTestSuite(),
        doctest.DocFileSuite('../README.txt',
                             optionflags=doctest.ELLIPSIS),
    ])Test groupingIn addition to per-package and per-module filtering, zope.testrunner
has other mechanisms for grouping tests:layersallow you to have shared setup/teardown code to be used
by a group of tests, that is executed only once, and not for each
test.  Layers are orthogonal to the usual package/module structure
and are specified by setting thelayerattribute on test
suites.levelsallow you to group slow-running tests and not run them
by default.  They’re specified by setting thelevelattribute
on test suites to an int.Other featureszope.testrunner can profile your tests, measure test coverage,
check for memory leaks, integrate withsubunit, shuffle the
test execution order, and run multiple tests in parallel.zope.testrunner Changelog6.4 (2024-02-27)Add PEP 440 support (implicit namespaces).
(#160)6.3.1 (2024-02-12)Fix XML tests when running in distribution resp. separately.
(#163)6.3 (2024-02-07)Exit cleanly when using the test runner--versionargument.
(#102)Add new--xml<path>option to write JUnit-like XML reports.
Code comes fromcollective.xmltestreport, but be aware that here--xmlis not a boolean, but expects a path!
(#148).Add support for Python 3.13a3.6.2.1 (2023-12-22)Work around Python 3.12.1+ no longer callingstartTestfor skipped tests
(#157).6.2 (2023-11-08)Add support for Python 3.12.Update code and tests topython-subunit>= 1.4.3thus requiring at least
this version.6.1 (2023-08-26)Add preliminary support for Python 3.12b4.
(#149)6.0 (2023-03-28)Drop support for Python 2.7, 3.5, 3.6.5.6 (2022-12-09)Add support for Python 3.11.Inline a small part ofrandom.Random.shufflewhich was deprecated in
Python 3.9 and removed in 3.11 (#119).Don’t trigger post mortem debugger for skipped tests. (#141).5.5.1 (2022-09-07)Fix: let--at-level=levelwithlevel <= 0run the tests
at all levels (rather than at no level)#138.5.5 (2022-06-24)Usesys._current_frames(rather thanthreading.enumerate)
as base for new thread detection, fixes#130.New option--gc-after-test. It calls for a garbage collection
after each test and can be used to track downResourceWarning``sand cyclic garbage. With ``rv = gc.collect(),!is output on verbosity level 1 whenrvis non zero (i.e. when cyclic structures have been released),[``*rv*]`` on higher verbosity levels and
a detailed cyclic garbage analysis on verbosity level 4+.
For details, see#133.Allow the filename for the logging configuration to be specified
via the envvarZOPE_TESTRUNNER_LOG_INI.
If not defined, the configuration continues to be locked for
in filelog.iniof the current working directory.
Remember the logging configuration file in envvarZOPE_TESTRUNNER_LOG_INIto allow spawned child processes
to recreate the logging configuration.
For details, see#134.5.4.0 (2021-11-19)Improve--helpdocumentation for--package-pathoption
(#121).Do not disable existing loggers during logsupport initialization
(#120).Fix tests with testtools >= 2.5.0 (#125).Add support for Python 3.10.5.3.0 (2021-03-17)Add support for Python 3.9.Fixpackage init file missingwarning
(#112).Make standard streams provide abufferattribute on Python 3 when using–bufferor testing under subunit.5.2 (2020-06-29)Add support for Python 3.8.When a layer is run in a subprocess, read its stderr in a thread to avoid
a deadlock if its stderr output (containing failing and erroring test IDs)
overflows the capacity of a pipe (#105).5.1 (2019-10-19)Recover more gracefully when layer setUp or tearDown fails, producing
useful subunit output.Prevent a spurious warning from the--require-uniqueoption if the--moduleoption was not used.Add optional buffering of standard output and standard error during tests,
requested via the--bufferoption or enabled by default for subunit.Fix incorrect failure counts in per-layer summary output, broken in 4.0.1.5.0 (2019-03-19)Fix test failures and deprecation warnings occurring when using Python 3.8a1.
(#89)Drop support for Python 3.4.4.9.2 (2018-11-24)FixTypeError: abytes-likeobject is required, not 'str'running tests in parallel on Python 3. Seeissue 80.4.9.1 (2018-11-21)Fix AssertionError in _DummyThread.isAlive on Python 3 (#81).4.9 (2018-10-05)Drop support for Python 3.3.Add support for Python 3.7.Enable test coverage reporting on coveralls.io and in tox.ini.Host documentation athttps://zopetestrunner.readthedocs.ioRemove untested support for the--pycheckeroption. Seeissue 63.Update the command line interface to useargparseinstead ofoptparse. Seeissue 61.Use ipdb instead of pdb for post-mortem debugging if available
(#10).Add a –require-unique option to check for duplicate test IDs. SeeLP #682771.Reintroduce optional support forsubunit, now with support for both
version 1 and version 2 of its protocol.Handle string in exception values when formatting chained exceptions.
(#74)4.8.1 (2017-11-12)EnableDeprecationWarningearlier, when discovering test
modules. This lets warnings that are raised on import (such as those
produced byzope.deprecation.moved) be reported. Seeissue 57.4.8.0 (2017-11-10)Automatically enableDeprecationWarningwhen running tests. This
is recommended by the Python core developers and matches the
behaviour of theunittestmodule. This can be overridden with
Python command-line options (-W) or environment variables
(PYTHONWARNINGS). Seeissue 54.4.7.0 (2017-05-30)Drop all support forsubunit.4.6.0 (2016-12-28)Make thesubunitsupport purely optional: applications which have
been getting the dependencies viazope.testrunnershould either addzope.testrunner[subunit]to theirinstall_requiresor else
depend directly onpython-subunit.New option--ignore-new-thread=<regexp>to suppress “New thread(s)”
warnings.Support Python 3.6.4.5.1 (2016-06-20)Fixed: Using the-joption to run tests in multiple processes
caused tests that used themultiprocessingpackage to hang
(because the testrunner replacedsys.stdinwith an unclosable
object).Drop conditional dependency onunittest2(redundant after dropping
support for Python 2.6).4.5.0 (2016-05-02)Stop tests for all layers when test fails/errors when started with
-x/–stop-on-error
(#37).Drop support for Python 2.6 and 3.2.4.4.10 (2015-11-10)Add support for Python 3.5
(#31).Insert extra paths (from--path) to the front of sys.argv
(#32).4.4.9 (2015-05-21)When using-j, parallelize all the tests, including the first test layer
(#28).4.4.8 (2015-05-01)Support skipped tests in subunit output
(#25).More efficient test filtering
(#26).4.4.7 (2015-04-02)Work around a bug in PyPy3’s curses module
(#24).4.4.6 (2015-01-21)Restore support for instance-based test layers that regressed in 4.4.5
(#20).4.4.5 (2015-01-06)Sort related layers close to each other to reduce the number of unnecessary
teardowns (fixes#14).Run the unit test layer first (fixesLP #497871).4.4.4 (2014-12-27)When looking for the right location of test code, start with longest
location paths first. This fixes problems with nested code locations.4.4.3 (2014-03-19)Added support for Python 3.4.4.4.2 (2014-02-22)Drop support for Python 3.1.Fix post-mortem debugging when a non-printable exception happens
(https://github.com/zopefoundation/zope.testrunner/issues/8).4.4.1 (2013-07-10)Updatedboostrap.pyto version 2.2.Fix nondeterministic test failures on Python 3.3Tear down layers afterpost_mortemdebugging is finished.Fix tests that write to source directory, it might be read-only.4.4.0 (2013-06-06)Fix tests selection when the negative “!” pattern is used several times
(LP #1160965)Moved tests into a ‘tests’ subpackage.Madepython-mzope.testrunnerwork again.Support ‘skip’ feature of unittest2 (which became the new unittest in Python
2.7).Better diagnostics when communication with subprocess fails
(https://github.com/zopefoundation/zope.testrunner/issues/5).Do not break subprocess execution when the test suite changes the working
directory (https://github.com/zopefoundation/zope.testrunner/issues/6).Count test module import errors as errors (LP #1026576).4.3.3 (2013-03-03)Running layers in sub-processes did not use to work when run viapython setup.py ftestsince it tried to run setup.py with all the
command line options. It now detectssetup.pyruns and we run the test
runner directly.4.3.2 (2013-03-03)FixSkipLayersclass in cases where the distribution specifies atest_suitevalue.4.3.1 (2013-03-02)Fixed a bug in theftestcommand and added a test.Fixed a trivial test failure with Python 3 of the previous release.4.3.0 (2013-03-02)Exposeftestdistutils command via an entry point.Added tests forzope.testrunner.eggsupport.4.2.0 (2013-02-12)Dropped use of 2to3, rewrote source code to be compatible with all Python
versions.  Introduced a dependency onsix.4.1.1 (2013-02-08)Dropped use of zope.fixers (LP: #1118877).Fixed tox test error reporting; fixed tests on Pythons 2.6, 3.1, 3.2, 3.3 and
PyPy 1.9.Fix –shuffle ordering on Python 3.2 to be the same as it was on older Python
versions.Fix –shuffle nondeterminism when multiple test layers are present.
Note: this will likely change the order of tests for the same –shuffle-seed.New option: –profile-directory.  Use it in the test suite so that tests
executed by detox in parallel don’t conflict.Use a temporary coverage directory in the test suite so that tests
executed by detox in parallel don’t conflict.Fix –post-mortem (aka -D, –pdb) when a test module cannot be imported
or is invalid (LP #1119363).4.1.0 (2013-02-07)Replaced deprecatedzope.interface.implementsusage with equivalentzope.interface.implementerdecorator.Dropped support for Python 2.4 and 2.5.Made StartUpFailure compatible with unittest.TextTestRunner() (LP #1118344).4.0.4 (2011-10-25)Work around sporadic timing-related issues in the subprocess buffering
tests.  Thanks to Jonathan Ballet for the patch!4.0.3 (2011-03-17)Added back support for Python <= 2.6 which was broken in 4.0.2.4.0.2 (2011-03-16)Added back Python 3 support which was broken in 4.0.1.FixedUnexpected successsupport by implementing the whole concept.Added support for the new __pycache__ directories in Python 3.2.4.0.1 (2011-02-21)LP #719369: AnUnexpected success(concept introduced in Python 2.7) is
no longer handled as success but as failure. This is a workaround. The
whole unexpected success concept might be implemented later.4.0.0 (2010-10-19)Show more information about layers whose setup fails (LP #638153).4.0.0b5 (2010-07-20)Update fix for LP #221151 to a spelling compatible with Python 2.4.Timestamps are now always included in subunit output (r114849).LP #591309: fix a crash when subunit reports test failures containing
UTF8-encoded data.4.0.0b4 (2010-06-23)Package as a zipfile to work around Python 2.4 distutils bug (no
feature changes or bugfixes inzope.testrunneritself).4.0.0b3 (2010-06-16)LP #221151: keepunittest.TestCase.shortDescriptionhappy by supplying
a_testMethodDocattribute.LP #595052: keep the distribution installable under Python 2.4:  its
distutils appears to munge the empty__init__.pyfile in thefoo.baregg used for testing into a directory.LP #580083: fix thebin/testscript to run only tests fromzope.testrunner.LP #579019: When layers were run in parallel, their tearDown was
not called. Additionally, the first layer which was run in the main
thread did not have its tearDown called either.4.0.0b2 (2010-05-03)Having ‘sampletests’ in the MANIFEST.in gave warnings, but doesn’t actually
seem to include any more files, so I removed it.Moved zope.testing.exceptions to zope.testrunner.exceptions. Now
zope.testrunner no longer requires zope.testing except for when running
its own tests.4.0.0b1 (2010-04-29)Initial release of the testrunner from zope.testrunner as its own module.
(Previously it was part of zope.testing.)