Package: test-exe-matrix
A wrapper around py.test to run UNIX commands and check some of their behaviour.ContentsInstallation and runningMain requirementsFrom PyPA (pip install)InstallRunFrom source with poetryInstallRunCommand line optionsParametrizing testsMinimum viable testOptional valuesExampleDev: Build the packageOther packagingLicenceThank youInstallation and runningMain requirementsUNIX likePython 3.6, 3.7 or 3.8From PyPA (pip install)InstallThe most recent packaged version can be installed withpipwith options relevant to your usecase[1].[1]For instancepip{3} install--usertest_exe_matrix, or install in a virtualenv.pip install test_exe_matrixRuntest_exe_matrix [arguments...]From source with poetryInstallNote on poetry: you can install it from pip (pip install--userpoetry)[2]or followthe instructionson theofficial Web site.[2]They say::
Be aware that it will also install Poetry’s dependencies which might cause conflicts with other packages.git clone https://framagit.org/feth/test_exe_matrix.git
cd test_exe_matrix
poetry installRunpoetry run test_exe_matrix [arguments...]Command line optionsTypical use:[poetry run] test_exe_matrixSynopsys:usage: test_exe_matrix [-h] [-V] [--collect-only] [-v] [--markers]
                       [-m PYTEST_ARGS]
                       [testsuite [testsuite ...]]

Test command lines listed in yaml files.

positional arguments:
  testsuite       testsuite yaml file -see example
                  test_exe_matrix/matrix.yaml.
                  Argument can be specified multiple times

optional arguments:
  -h, --help      show this help message and exit
  -V, --version   show program's version number and exit
  --collect-only  Only collect tests, without running them.
  -v              Increase verbosity.
  --markers       Lists registered test markers.
  -m PYTEST_ARGS  Only run tests that match selection expression, ie. '-m "not
                  internet"'

test_exe_matrix is mainly a Py.test wrapper-v adds very nice colors, thanks to py.test.Parametrizing testsPut your test suites in a yaml, like matrix.yaml (provided), or in several. Every yaml file will have a separate global config (markers declaration and general timeout).Minimum viable test- exe: /path/to/exeThe test will succeed if default values are met: see below (optional values).Optional valuestimeoutAllowed running time in seconds. Program will be killed after this delay. Default to config/timeout or 1.retcodeExpected return code.stdoutExpected standard output (full match)stderrExpected standard error (full match)partstdoutSubstring expected in standard output.partstderrSubstring expected in standard error.notinstdoutSubstring is not expected in standard output.notinstderrSubstring is not expected in standard error.argsList of arguments to your program.nameDescription of the test being made.expect_too_longBoolean: whether the program is expected to run over the time limit.markersList of markers for the test. You can use standard py.test markers (especially ‘xfail’: indicating ‘expected failure’) or custom markers, that _must_ be declared in the config section.ExampleA complete (hopefully) example file is provided as “matrix.yaml”.Given the examples, you may runtest_exe_matrix matrix.yaml --collect-onlyto list the tests, andtest_exe_matrix matrix.yaml -v -m "not internet and not slow"to deselect all tests requiring Internet and slow tests.Dev: Build the packageThe project uses poetry. Dopoetry buildAnd a pip installable wheel package will appear under dist/. For
instance, you can install with:pip3 install dist/test_exe_matrix-0.0.18-py3-none-any.whlOther packagingDebian/Fedora/Conda packages would be a good thing, but I don’t know where to start. If I have to drop Poetry, let it be. Help welcome.LicenceDual licensed (at your option) to the public under CeCILL 2.1 and GPL 2, see files LICENSE_CeCILL_2.1.txt and LICENSE_GPL_2.txt.Those are compatible copyleft licenses; CeCILL 2.1 is better suited for France at least.Other needs: contact me.Thank youStéphane Bortzmeyer, who once asked if a simple tool testing the result of commands existed, and then reported bugs/wrote helpful tickets on framagit/gitlab to guide the development.