Package: pytest-console-scripts
pytest-console-scriptsPytest-console-scripts is apytestplugin for running python scripts from
within tests. It's quite similar tosubprocess.run(), but it also has an
in-process mode, where the scripts are executed by the interpreter that's
runningpytest(using some amount of sandboxing).In-process mode significantly reduces the run time of the test suites that
run many external scripts. This is speeds up development. In the CI environment
subprocess mode can be used to make sure the scripts also work (and behave the
same) when run by a fresh interpreter.RequirementsPython 3.8+, or PyPy3,Pytest 4.0 or newer.InstallationYou can install "pytest-console-scripts" viapipfromPyPI:$pipinstallpytest-console-scriptsNormally you would add it as a test dependency intox.ini(seetox
documentation).UsageThis plugin will run scripts that are installed viaconsole_scriptsentry
point insetup.py, python files in current directory (or anywhere else, if
given the path), and Python scripts anywhere else in the path. It will also run
executables that are not Python scripts, but only in subprocess mode (there's
no benefit in usingpytest-console-scriptsfor this, you should just usesubprocess.run).Here's an example withconsole_scriptsentry point. Imagine we have a python
packagefoowith the followingsetup.py:setup(name='foo',version='0.0.1',py_modules=['foo'],entry_points={'console_scripts':['foobar=foo:bar']},)We could use pytest-console-scripts to test thefoobarscript:deftest_foo_bar(script_runner):result=script_runner.run(['foobar','--version'])assertresult.returncode==0assertresult.stdout=='3.2.1\n'assertresult.stderr==''script_runner.run('foobar --version',shell=True,check=True)This would use thescript_runnerfixture provided by the plugin to
run the script and capture its output.The arguments ofscript_runner.runare the command name of the script and
any command line arguments that should be passed to it. Additionally the
following keyword arguments can be used:cwd- set the working directory of the script under test.env- a dictionary with environment variables to use instead of the current
environment.stdin- a file-like object that will be piped to standard input of the
script.check- raises an exception ifreturncode != 0, defaults to False.shell- mimic shell execution, this should work well for simple cases,
defaults to False.Type-hinting is also supported.
You may type-hint the fixture with the following code:frompytest_console_scriptsimportScriptRunnerdeftest_foo_bar(script_runner:ScriptRunner)->None:...Configuring script execution modeIn the example above thefoobarscript would run in in-process mode (which is
the default). This is fast and good for quick iteration during development.
After we're happy with the functionality, it's time to run the script in
subprocess mode to simulate real invocation more closely. There are several
ways to do this. We can configure it via pytest configuration (for example intox.ini):[pytest]script_launch_mode=subprocessWe can give a command line option to pytest (this will override the
configuration file):$pytest--script-launch-mode=subprocesstest_foobar.pyWe can also mark individual tests to run in a specific mode:@pytest.mark.script_launch_mode('subprocess')deftest_foobar(script_runner):...Between these three methods the marking of the tests has priority before the
command line option that in turn overrides the configuration setting. All three
can take three possible values: "inprocess", "subprocess", and "both" (which
will cause the test to be run twice: in in-process and in subprocess modes).Interaction with mockingIt is possible to mock objects and functions inside of console scripts when
they are run usingpytest-console-scriptsbut only in inprocess mode. When
the script is run in subprocess mode, it is executed by a separate Python
interpreter and the test can't mock anything inside of it.Another limitation of mocking is that with simple Python scripts that are not
installed viaconsole_scriptsentry pointmocking of objects inside of
the main script will not work. The reason for that is that when we runmyscript.pywith$ python myscript.pythe script gets imported into__main__namespace instead ofmyscriptnamespace. Our patching ofmyscript.myfunctionwill have no effect on what the code in__main__namespace sees when it's callingmyfunctiondefined in the same file.Seethis stackoverflow answerfor some ideas of how to get around this.Suppressing the printing of script run resultsWhen tests involvingpytest-console-scriptsfail, it tends to be quite
useful to see the output of the scripts that were executed in them. We try
to be helpful and print it out just before returning the result fromscript_runner.run(). Normally PyTestcapturesall the output during a
test run and it's not shown to the user unless some tests fail. This is exactly
what we want.However, in some cases it might be useful to disable the output capturing and
PyTest providesways to do it. When capturing is disabled, all test run
results will be printed out and this might make it harder to inspect the other
output of the tests. To deal with this,pytest-console-scriptshas an option
to disable the printing of script run results:$pytest--hide-run-resultstest_foobar.pyIt's also possible to disable it just for one script run:result=script_runner.run('foobar',print_result=False)When printing of script run results is disabled, script output won't be
visible even when the test fails. Unfortunately there's no automatic way to
print it only if the test fails because by the time a script run completes we
don't know whether the test will fail or not. It's possible to do it manually
from the test by using:result.print()This, combined with--hide-run-resultsorprint_result=Falsecan be used to
only print interesting run results when capturing is off.Package installation and testing during developmentSincepytest-console-scriptsrelies on the scripts being located in the path,
it can only run the console scripts from packages that have been installed (if
you are interested in working on removing this limitation, take a look atthis
ticketand in
particularthis comment).
If you want to run the tests quickly during development, the additional
installation step would add a significant overhead and slow you down.There's a way around this: install your package indevelopment modeusingpip install -e .. If you usetox, you can take one of its
existing virtualenvs (they live in.tox/). Otherwise create avirtualenvjust for development, activate it and runpython setup.py developto install your package in development mode. You will need to
re-install every time you add a new console script, but otherwise all the
changes to your code will be immediately picked up by the tests.ContributingContributions are very welcome. Tests can be run withtox, please ensure
the coverage at least stays the same before you submit a pull request.LicenseDistributed under the terms of theMITlicense, "pytest-console-scripts"
is free and open source software.IssuesIf you encounter any problems, pleasefile an issuealong with a detailed
description.Pytest-console-scripts was initially generated withCookiecutteralong
with@hackebrot'sCookiecutter-pytest-plugintemplate.