Package: pytest-quarantine
pytest-quarantineA plugin forpytestto manage expected test failures.FeaturesSave the list of failing tests, so that they can be automatically marked as expected failures on future test runs.Why?You've got a test suite; nicely done! Unfortunately, for completely understandable reasons, a lot of the tests are failing. Someday, you and/or your team will get those tests to pass. For now, though, what you really want is to draw a line in the sand and avoid new test failures. Unfortunately, it's hard to tell when they're introduced because the test suite is already failing. You could get the test suite to pass by applyingpytest.mark.xfailto the existing failures, but there areso manyof them.With this plugin, you can save all of existing failures to a file (the quarantine). On future test runs, this plugin will automatically applypytest.mark.xfailto the quarantined tests. Then, the test suite will pass, and any new failures will cause it to fail.RequirementsPython 2.7 or 3.5+pytest 4.6 or newerInstallationViapipfromPyPIin an activevirtual environment:$ pip install pytest-quarantineUsageRun your test suite and save the failing tests toquarantine.txt:$ pytest --save-quarantine=quarantine.txt
============================= test session starts ==============================
...
collected 1380 items

...

---------------------- 661 items saved to quarantine.txt -----------------------
============== 629 failed, 719 passed, 32 error in 312.56 seconds ==============Addquarantine.txtto your version control system.Run your test suite with the quarantined tests marked as expected failures:$ pytest --quarantine=quarantine.txt
============================= test session starts ==============================
...
collected 1380 items
added mark.xfail to 661 of 661 items from quarantine.txt

...

================== 719 passed, 661 xfailed in 300.51 seconds ===================When the expected failures eventually pass (i.e., they get counted asxpassed), they can be removed manually fromquarantine.txt, or automatically using--save-quarantine. Note that the latter will overwrite the contents of the quarantine, so it's best to only use it when running the entire test suite.Getting helpPlease submit questions, bug reports, and feature requests in theissue tracker.ContributingImprovements to the code and documentation are greatly appreciated. SeeHow to contributefor details.Code of conductEveryone interacting with this project is expected to follow theContributor Covenant.LicenseDistributed under the terms of theMITlicense.AcknowledgementsThis project was initially developed atEnergySageto aid our migration to Python 3. We hope other people find it helpful.The name was inspired by thequarantine featureof the Bamboo CI/CD service.Pros and Cons of Quarantined Testsis a good introduction to the concept (not related to this project).The repository was generated withCookiecutterusing thecookiecutter-pytest-plugintemplate. The layout and tooling has been heavily modified since then, but it was very helpful to start.Some other helpful resources were:Starting an Open Source Project | Open Source GuidesTesting & Packaging | Hynek SchlawackMaintaining a Python Project When It’s Not Your Job | Hynek SchlawackPython Testing with pytest | Brian Okken | The Pragmatic BookshelfDeveloping better test suites for pytest plugins | Raphael PierzinaGitHub searchfor examples fromorg:pytest-dev org:pypa org:pycqa org:pallets org:encode