Package: pytest-pipeline
pytest-pipelinepytest-pipeline is a pytest plugin for functional testing of data analysis
pipelines. They are usually long-running scripts or executables with multiple
input and/or output files + directories. The plugin is meant for end-to-end
testing where you test for conditions before the pipeline run and after the
pipeline runs (output files, checksums, etc.).pytest-pipeline is tested against Python versions 2.7, 3.3, 3.4, 3.5, and 3.6.Installationpip install pytest-pipelineWalkthroughFor our example, we will use a super simple pipeline that writes a file and
prints to stdout:#!/usr/bin/env pythonfrom__future__importprint_functionif__name__=="__main__":withopen("result.txt","w")asresult:result.write("42\n")print("Result computed")At this point it’s just a simple script, but it should be enough to illustrate
the plugin. Also, if you want to follow along, save the above file asrun_pipeline.With the pipeline above, here’s how your test would look like withpytest_pipeline:importosimportshutilimportunittestfrompytest_pipelineimportPipelineRun,mark,utils# we can subclass `PipelineRun` to add custom methods# using `PipelineRun` as-is is also possibleclassMyRun(PipelineRun):# before_run-marked functions will be run before the pipeline is executed@mark.before_rundeftest_prep_executable(self):# copy the executable to the run directoryshutil.copy2("/path/to/run_pipeline","run_pipeline")# ensure that the file is executableassertos.access("run_pipeline",os.X_OK)# a pipeline run is treated as a test fixturerun=MyRun.class_fixture(cmd="./run_pipeline",stdout="run.stdout")# the fixture is bound to a unittest.TestCase using the usefixtures mark@pytest.mark.usefixtures("run")# tests per-pipeline run are grouped in one unittest.TestCase instanceclassTestMyPipeline(unittest.TestCase):deftest_result_md5(self):assertutils.file_md5sum("result.txt")=="50a2fabfdd276f573ff97ace8b11c5f4"deftest_exit_code(self):# the run fixture is stored as the `run_fixture` attributeassertself.run_fixture.exit_code==0# we can check the stdout that we capture as welldeftest_stdout(self):assertopen("run.stdout","r").read().strip()=="Result computed"If the test above is saved astest_demo.py, you can then run the test by
executingpy.test-vtest_demo.py. You should see that three tests were
executed and all four passed.What just happened?You just executed your first pipeline test. The plugin itself gives you:Test directory creation (one class gets one directory).
By default, testdirectories are all created in the/tmp/pipeline_testdirectory. You can tweak this location by supplying the--base-pipeline-dircommand line flag.Automatic execution of the pipeline.
No need toimport subprocess, just define the command via thePipelineRunobject. We optionally captured the standard output to a file
calledrun.stdout. Not a fan of doing disk IO? You can also setstdoutand/orstderrtoTrueand have their values captured in-memory.Timeout control.
For long running pipelines, you can also supply atimeoutargument which
limits how long the pipeline process can run.And since this is a py.test plugin, test discovery and execution is done via
py.test.Getting + giving helpPlease use theissue trackerto report bugs or feature requests. You can always fork and submit a pull
request as well.LicenseSee LICENSE.ChangelogVersion 0.3Release 0.3.0Release date: 24 January 2017Allow stdout and/or stderr capture in-memory. This can be done by
setting their respective keyword arguments toTruewhen creating
the run fixture.Version 0.2Release 0.2.0Release date: 31 March 2015Pipeline runs are now modelled differently. Instead of a class attribute,
they are now created as pytest fixtures. This allows the pipeline runs
to be used in non-unittest.TestCasetests.Theafter_rundecorator is deprecated.The command line flags–xfail-pipelineand–skip-runare deprecated.Version 0.1Release 0.1.0Release date: 25 August 2014First release on PyPI.