neulab/tldr huggingface dataset:
Whenever you change traits that are not tracked we recommend you to manually trigger a rebuild of the tox environment by passing the -r flag for the tox invocation. For instance, for a setuptools project whenever you modify the install_requires keyword at the next run force the recreation of the tox environment by passing the recreate cli tox flag.
Selecting one or more environments to run tests against Using the -e ENV[,ENV36,...]  option you explicitly list the environments where you want to run tests against. For example, given the previous sphinx example you may call:
which will make tox only manage the docs environment and call its test commands.  You may specify more than one environment like this:
tox -e py27,py36
which would run the commands of the py27 and py36 testenvironments respectively.  The special value ALL selects all environments.
You can also specify an environment list in your tox.ini:
[tox] envlist = py27,py36
or override it from the command line or from the environment variable TOXENV:
export TOXENV=py27,py36 # in bash style shells
Access package artifacts between multiple tox-runs If you have multiple projects using tox you can make use of a distshare directory where tox will copy in sdist-packages so that another tox run can find the "latest" dependency.  This feature allows to test a package against an unreleased development version or even an uncommitted version on your own machine.
By default, {homedir}/.tox/distshare will be used for copying in and copying out artifacts (i.e. Python packages).
For project two to depend on the one package you use the following entry:
# example two/tox.ini [testenv] # install latest package from "one" project deps = {distshare}/one-*.zip
That's all.  tox running on project one will copy the sdist-package into the distshare directory after which a tox run on project two will grab it because deps contain an entry with the one-*.zip pattern.  If there is more than one matching package the highest version will be taken.  tox uses verlib  to compare version strings which must be compliant with PEP 386.
If you want to use this with Jenkins, also checkout the jenkins artifact example.
basepython defaults, overriding For any pyXY test environment name the underlying pythonX.Y executable will be searched in your system PATH. Similarly, for jython and pypy the respective jython and pypy-c names will be looked for.  The executable must exist in order to successfully create virtualenv environments. On Windows a pythonX.Y named executable will be searched in typical default locations using the C:\PythonX.Y\python.exe pattern.
All other targets will use the system python instead. You can override any of the default settings by defining the basepython variable in a specific test environment section, for example:
[testenv:docs] basepython = python2.7
Avoiding expensive sdist Some projects are large enough that running an sdist, followed by an install every time can be prohibitively costly. To solve this, there are two different options you can add to the tox section. First, you can simply ask tox to please not make an sdist:
[tox] skipsdist=True
If you do this, your local software package will not be installed into the virtualenv. You should probably be okay with that, or take steps to deal with it in your commands section:
[testenv] commands = python setup.py develop pytest
Running setup.py develop is a common enough model that it has its own option:
[testenv] usedevelop=True
And a corresponding command line option --develop, which will set skipsdist to True and then perform the setup.py develop step at the place where tox normally performs the installation of the sdist.  Specifically, it actually runs pip install -e . behind the scenes, which itself calls setup.py develop.
There is an optimization coded in to not bother re-running the command if $projectname.egg-info is newer than setup.py or setup.cfg.
Understanding InvocationError exit codes When a command (defined by commands = in tox.ini) fails, it has a non-zero exit code, and an InvocationError exception is raised by tox:
ERROR: InvocationError for command '<command defined in tox.ini>' (exited with code 1)
If the command starts with pytest or python setup.py test for instance, then the pytest exit codes are relevant.
On unix systems, there are some rather common exit codes.  This is why for exit codes larger than 128, if a signal with number equal to <exit code> - 128 is found in the signal module, an additional hint is given:
ERROR: InvocationError for command '<command>' (exited with code 139) Note: this might indicate a fatal error signal (139 - 128 = 11: SIGSEGV)
where <command> is the command defined in tox.ini, with quotes removed.
The signal numbers (e.g. 11 for a segmentation fault) can be found in the "Standard signals" section of the signal man page.  Their meaning is described in POSIX signals.
Beware that programs may issue custom exit codes with any value, so their documentation should be consulted.
Sometimes, no exit code is given at all.  An example may be found in pytest-qt issue #170, where Qt was calling abort() instead of exit().
SEE ALSO: ignoring exit code.
Using tox with the Jenkins Integration Server Using Jenkins multi-configuration jobs The Jenkins continuous integration server allows to define "jobs" with "build steps" which can be test invocations.  If you install tox on your default Python installation on each Jenkins slave, you can easily create a Jenkins multi-configuration job that will drive your tox runs from the CI-server side, using these steps:
• install the Python plugin for Jenkins under "manage jenkins"
• create a "multi-configuration" job, give it a name of your choice
• configure your repository so that Jenkins can pull it
• (optional) configure multiple nodes so that tox-runs are performed on multiple hosts
• configure axes by using TOXENV as an axis name and as values provide space-separated test environment names you want Jenkins/tox to execute.
• add a Python-build step with this content (see also next example):
import tox
os.chdir(os.getenv("WORKSPACE")) tox.cmdline()  # environment is selected by ``TOXENV`` env variable
• check Publish JUnit test result report and enter **/junit-*.xml as the pattern so that Jenkins collects test results in the JUnit XML format.
The last point requires that your test command creates JunitXML files, for example with pytest it is done like this:
[testenv] commands = pytest --junitxml=junit-{envname}.xml
zero-installation for slaves NOTE: This feature is broken currently because "toxbootstrap.py" has been removed.  Please file an issue if you'd like to see it back.
If you manage many Jenkins slaves and want to use the latest officially released tox (or latest development version) and want to skip manually installing tox then substitute the above Python build step code with this:
import urllib, os
url = "https://bitbucket.org/hpk42/tox/raw/default/toxbootstrap.py" # os.environ['USETOXDEV']="1"  # use tox dev version d = dict(__file__="toxbootstrap.py") exec urllib.urlopen(url).read() in d d["cmdline"](["--recreate"])
The downloaded toxbootstrap.py file downloads all necessary files to install tox in a virtual sub environment.  Notes:
• uncomment the line containing USETOXDEV to use the latest development-release version of tox instead of the latest released version.
• adapt the options in the last line as needed (the example code will cause tox to reinstall all virtual environments all the time which is often what one wants in CI server contexts)
Integrating sphinx documentation checks in a Jenkins job If you are using a multi-configuration Jenkins job which collects JUnit Test results you will run into problems using the previous method of running the sphinx-build command because it will not generate JUnit results.  To accommodate this issue one solution is to have pytest wrap the sphinx-checks and create a JUnit result file which wraps the result of calling sphinx-build.  Here is an example:
1. create a docs environment in your tox.ini file like this:
[testenv:docs] basepython = python # change to ``doc`` dir if that is where your sphinx-docs live changedir = doc deps = sphinx py commands = pytest --tb=line -v --junitxml=junit-{envname}.xml check_sphinx.py
2. create a doc/check_sphinx.py file like this:
import py import subprocess
def test_linkcheck(tmpdir): doctrees = tmpdir.join("doctrees") htmldir = tmpdir.join("html") subprocess.check_call( ["sphinx-build", "-W", "-blinkcheck", "-d", str(doctrees), ".", str(htmldir)] )
def test_build_docs(tmpdir): doctrees = tmpdir.join("doctrees") htmldir = tmpdir.join("html") subprocess.check_call( ["sphinx-build", "-W", "-bhtml", "-d", str(doctrees), ".", str(htmldir)] )
3. run tox -e docs and then you may integrate this environment along with your other environments into Jenkins.
Note that pytest is only installed into the docs environment and does not need to be in use or installed with any other environment.
Access package artifacts between Jenkins jobs In an extension to artifacts you can also configure Jenkins jobs to access each others artifacts.  tox uses the distshare directory to access artifacts and in a Jenkins context (detected via existence of the environment variable HUDSON_URL); it defaults to to {toxworkdir}/distshare.
This means that each workspace will have its own distshare directory and we need to configure Jenkins to perform artifact copying.  The recommend way to do this is to install the Jenkins Copy Artifact plugin and for each job which "receives" artifacts you add a Copy artifacts from another project build step using roughly this configuration:
Project-name: name of the other (tox-managed) job you want the artifact from Artifacts to copy: .tox/dist/*.zip   # where tox jobs create artifacts Target directory: .tox/distshare     # where we want it to appear for us Flatten Directories: CHECK           # create no subdir-structure
You also need to configure the "other" job to archive artifacts; This is done by checking Archive the artifacts and entering:
Files to archive: .tox/dist/*.zip
So our "other" job will create an sdist-package artifact and the "copy-artifacts" plugin will copy it to our distshare area.  Now everything proceeds as artifacts shows it.
So if you are using defaults you can re-use and debug exactly the same tox.ini file and make use of automatic sharing of your artifacts between runs or Jenkins jobs.
Avoiding the path too long error with long shebang lines When using tox on a Jenkins instance, there may be a scenario where tox can not invoke pip because the shebang (Unix) line is too long. Some systems only support a limited amount of characters for an interpreter directive (e.x.  Linux as a limit of 128). There are two methods to workaround this issue:
1. Invoke tox with the --workdir option which tells tox to use a specific directory for its virtual environments. Using a unique and short path can prevent this issue.
2. Use the environment variable TOX_LIMITED_SHEBANG to deal with environments with interpreter directive limitations (consult long interpreter directives for more information).
Running tox environments in parallel Jenkins has parallel stages allowing you to run commands in parallel, however tox package building it is not parallel safe. Use the --parallel--safe-build flag to enable parallel safe builds (this will generate unique folder names for distdir, distshare and log.  Here's a generic stage definition demonstrating how to use this inside Jenkins:
stage('run tox envs') { steps { script { def envs = sh(returnStdout: true, script: "tox -l").trim().split('\n') def cmds = envs.collectEntries({ tox_env -> [tox_env, { sh "tox --parallel--safe-build -vve $tox_env" }] }) parallel(cmds) } } }
Development environment tox can be used for just preparing different virtual environments required by a project.
This feature can be used by deployment tools when preparing deployed project environments. It can also be used for setting up normalized project development environments and thus help reduce the risk of different team members using mismatched development environments.
Here are some examples illustrating how to set up a project's development environment using tox. For illustration purposes, let us call the development environment dev.
Example 1: Basic scenario Step 1 - Configure the development environment First, we prepare the tox configuration for our development environment by defining a [testenv:dev] section in the project's tox.ini configuration file:
[testenv:dev] basepython = python2.7 usedevelop = True
In it we state:
• what Python executable to use in the environment,
• that our project should be installed into the environment using setup.py develop, as opposed to building and installing its source distribution using setup.py install.
The development environment will reside in toxworkdir (default is .tox) just like the other tox environments.
We can configure a lot more, if we want to. For example, we can add the following to our configuration, telling tox not to reuse commands or deps settings from the base [testenv] configuration:
[testenv:dev] commands = deps =
Step 2 - Create the development environment Once the [testenv:dev] configuration section has been defined, we create the actual development environment by running the following:
tox -e dev
This creates the environment at the path specified by the environment's envdir configuration value.
Example 2: A more complex scenario Let us say we want our project development environment to:
• use Python executable python2.7,
• pull packages from requirements.txt, located in the same directory as tox.ini.
Here is an example configuration for the described scenario:
[testenv:dev] basepython = python2.7 usedevelop = True deps = -rrequirements.txt
Platform specification Basic multi-platform example Assuming the following layout:
[tox] # platform specification support is available since version 2.0 minversion = 2.0 envlist = py{27,36}-{mylinux,mymacos,mywindows}
[testenv] # environment will be skipped if regular expression does not match against the sys.platform string platform = mylinux: linux mymacos: darwin mywindows: win32
# you can specify dependencies and their versions based on platform filtered environments deps = mylinux,mymacos: py==1.4.32 mywindows: py==1.4.30
# upon tox invocation you will be greeted according to your platform commands= mylinux: python -c 'print("Hello, Linus!")' mymacos: python -c 'print("Hello, Steve!")' mywindows: python -c 'print("Hello, Bill!")'
you can invoke tox in the directory where your tox.ini resides.  tox creates two virtualenv environments with the python2.7 and python3.6 interpreters, respectively, and will then run the specified command according to platform you invoke tox at.
tox configuration specification configuration discovery At the moment tox supports three configuration locations prioritized in the following order:
1. pyproject.toml,
2. tox.ini,
3. setup.cfg.
As far as the configuration format at the moment we only support standard ConfigParser "ini-style" format (there is a plan to add a pure TOML one soon).  tox.ini and setup.cfg are such files. Note that setup.cfg requires the content to be under the tox:tox section. pyproject.toml on the other hand is in TOML format. However, one can inline the ini-style format under the tool.tox.legacy_tox_ini key as a multi-line string.
Below you find the specification for the ini-style format, but you might want to skim some examples first and use this page as a reference.
tox global settings Global settings are defined under the tox section as:
[tox] minversion = 3.4.0
minversion Define the minimal tox version required to run; if the host tox is less than this the tool with exit with an error message indicating the user needs to upgrade tox.
toxworkdir={toxinidir}/.tox(PATH) Directory for tox to generate its environments into, will be created if it does not exist.
temp_dir={toxworkdir}/.tmp(PATH) Directory where to put tox temporary files. For example: we create a hard link (if possible, otherwise new copy) in this directory for the project package. This ensures tox works correctly when having parallel runs (as each session will have its own copy of the project package - e.g.  the source distribution).
skipsdist=false(true|false) Flag indicating to perform the packaging operation or not. Set it to true when using tox for an application, instead of a library.
setupdir={toxinidir}(PATH) Indicates where the packaging root file exists (historically the setup.py for setuptools).  This will be the working directory when performing the packaging.
distdir={toxworkdir}/dist(PATH) Directory where the packaged source distribution should be put. Note this is cleaned at the start of every packaging invocation.
sdistsrc={toxworkdir}/dist(PATH) Do not build the package, but instead use the latest package available under this path.  You can override it via the command line flag --installpkg.
distshare={homedir}/.tox/distshare(PATH) Folder where the packaged source distribution will be moved, this is not cleaned between packaging invocations. On Jenkins (exists JENKINS_URL or HUDSON_URL environment variable) the default path is {toxworkdir}/distshare.
envlist(comma separated values) Determining the environment list that tox is to operate on happens in this order (if any is found, no further lookups are made):
• command line option -eENVLIST
• environment variable TOXENV
• tox.ini file's envlist
New in version 3.4.0: What tox environments are ran during the tox invocation can be further filtered via the operating system environment variable TOX_SKIP_ENV regular expression (e.g. py27.* means don't evaluate environments that start with the key py27). Skipped environments will be logged at level two verbosity level.
skip_missing_interpreters=config(config|true|false) New in version 1.7.2.
When skip missing interpreters is true will force tox to return success even if some of the specified environments were missing. This is useful for some CI systems or running on a developer box, where you might only have a subset of all your supported interpreters installed but don't want to mark the build as failed because of it. As expected, the command line switch always overrides this setting if passed on the invocation. Setting it to config means that the value is read from the config file.
ignore_basepython_conflict=false(true|false) New in version 3.1.0.
tox allows setting the python version for an environment via the basepython setting. If that's not set tox can set a default value from the environment name ( e.g. py37 implies Python 3.7). Matching up the python version with the environment name has became expected at this point, leading to surprises when some configs don't do so. To help with sanity of users a warning will be emitted whenever the environment name version does not matches up with this expectation. In a future version of tox, this warning will become an error.
Furthermore, we allow hard enforcing this rule (and bypassing the warning) by setting this flag to true. In such cases we ignore the basepython and instead always use the base python implied from the Python name. This allows you to configure basepython in the global testenv without affecting environments that have implied base python versions.
requires(LIST of PEP-508) New in version 3.2.0.
Specify python packages that need to exist alongside the tox installation for the tox build to be able to start. Use this to specify plugin requirements and build dependencies.
[tox] requires = tox-venv setuptools >= 30.0.0
NOTE: tox does not install those required packages for you. tox only checks if the requirements are satisfied and crashes early with an helpful error rather then later in the process.
isolated_build=false(true|false) New in version 3.3.0.
Activate isolated build environment. tox will use a virtual environment to build a source distribution from the source tree. For build tools and arguments use the pyproject.toml file as specified in PEP-517 and PEP-518. To specify the virtual environment Python version define use the isolated_build_env config section.
isolated_build_env=.package(string) New in version 3.3.0.
Name of the virtual environment used to create a source distribution from the source tree.
parallel_show_output=false(bool) New in version 3.7.0.
If set to True the content of the output will always be shown when running in parallel mode.
depends(comma separated values) New in version 3.7.0.
tox environments this depends on. tox will try to run all dependent environments before running this environment. Format is same as envlist (allows factor usage).
Jenkins override It is possible to override global settings inside a Jenkins instance ( detection is by checking for existence of the JENKINS_URL environment variable) by using the tox:jenkins section:
[tox:jenkins] commands = ...  # override settings for the jenkins context
tox environment settings Test environments are defined by a:
[testenv:NAME] commands = ...
section.  The NAME will be the name of the virtual environment.  Defaults for each setting in this section are looked up in the:
[testenv] commands = ...
testenv default section.
Complete list of settings that you can put into testenv* sections:
basepython(NAME-OR-PATH) Name or path to a Python interpreter which will be used for creating the virtual environment, this determines in practice the python for what we'll create a virtual isolated environment.  Use this to specify the python version for a tox environment. If not specified, the virtual environments factors (e.g. name part) will be used to automatically set one. For example, py37 means python3.7, py3 means python3 and py means python.
Changed in version 3.1: After resolving this value if the interpreter reports back a different version number than implied from the name a warning will be printed by default. However, if ignore_basepython_conflict is set, the value is ignored and we force the basepython implied from the factor name.
commands(ARGVLIST) The commands to be called for testing. Only execute if commands_pre succeed.
Each line is interpreted as one command; however a command can be split over multiple lines by ending the line with the \ character.
Commands will execute one by one in sequential fashion until one of them fails (their exit code is non-zero) or all of them succeed. The exit code of a command may be ignored (meaning they are always considered successful) by prefixing the command with a dash (-) - this is similar to how make recipe lines work. The outcome of the environment is considered successful only if all commands (these + setup + teardown) succeeded (exit code ignored via the - or success exit code value of zero).
Note   the virtual environment binary path (the bin folder within) is prepended to the os PATH, meaning commands will first try to resolve to an executable from within the virtual environment, and only after that outside of it. Therefore python translates as the virtual environments python (having the same runtime version as the basepython), and pip translates as the virtual environments pip.
commands_pre(ARGVLIST) New in version 3.4.
Commands to run before running the commands.  All evaluation and configuration logic applies from commands.
commands_post(ARGVLIST) New in version 3.4.
Commands to run after running the commands. Execute regardless of the outcome of both commands and commands_pre.  All evaluation and configuration logic applies from commands.
install_command=python -m pip install {opts} {packages}(ARGV) New in version 1.6.
Determines the command used for installing packages into the virtual environment; both the package under test and its dependencies (defined with deps).  Must contain the substitution key {packages} which will be replaced by the package(s) to install.  You should also accept {opts} if you are using pip -- it will contain index server options such as --pre (configured as pip_pre) and potentially index-options from the deprecated indexserver option.
list_dependencies_command=python -m pip freeze(ARGV) New in version 2.4.
The list_dependencies_command setting is used for listing the packages installed into the virtual environment.
ignore_errors=false(true|false) New in version 2.0.
If true, a non-zero exit code from one command will be ignored and further commands will be executed (which was the default behavior in tox < 2.0).  If false, then a non-zero exit code from one command will abort execution of commands for that environment.
It may be helpful to note that this setting is analogous to the -i or ignore-errors option of GNU Make. A similar name was chosen to reflect the similarity in function.
Note that in tox 2.0, the default behavior of tox with respect to treating errors from commands changed. tox < 2.0 would ignore errors by default. tox >= 2.0 will abort on an error by default, which is safer and more typical of CI and command execution tools, as it doesn't make sense to run tests if installing some prerequisite failed and it doesn't make sense to try to deploy if tests failed.
pip_pre=false(true|false) New in version 1.9.
If true, adds --pre to the opts passed to install_command. If install_command uses pip, this will cause it to install the latest available pre-release of any dependencies without a specified version. If false, pip will only install final releases of unpinned dependencies.
Passing the --pre command-line option to tox will force this to true for all testenvs.
Don't set this option if your install_command does not use pip.
whitelist_externals(MULTI-LINE-LIST) each line specifies a command name (in glob-style pattern format) which can be used in the commands section without triggering a "not installed in virtualenv" warning.  Example: if you use the unix make for running tests you can list whitelist_externals=make or whitelist_externals=/usr/bin/make if you want more precision. If you don't want tox to issue a warning in any case, just use whitelist_externals=* which will match all commands (not recommended).
changedir={toxinidir}(PATH) change to this working directory when executing the test command.
deps(MULTI-LINE-LIST) Test-specific dependencies - to be installed into the environment prior to project package installation.  Each line defines a dependency, which will be passed to the installer command for processing (see indexserver).  Each line specifies a file, a URL or a package name.  You can additionally specify an indexserver to use for installing this dependency but this functionality is deprecated since tox-2.3.  All derived dependencies (deps required by the dep) will then be retrieved from the specified indexserver:
[tox] indexserver = myindexserver = https://myindexserver.example.com/simple
[testenv] deps = :myindexserver:pkg
(Experimentally introduced in 1.6.1) all installer commands are executed using the {toxinidir} as the current working directory.
platform(REGEX) New in version 2.0.
A testenv can define a new platform setting as a regular expression.  If a non-empty expression is defined and does not match against the sys.platform string the test environment will be skipped.
setenv(MULTI-LINE-LIST) New in version 0.9.
Each line contains a NAME=VALUE environment variable setting which will be used for all test command invocations as well as for installing the sdist package into a virtual environment.
Notice that when updating a path variable, you can consider the use of variable substitution for the current value and to handle path separator.
[testenv] setenv   = PYTHONPATH = {env:PYTHONPATH}{:}{toxinidir}
passenv(SPACE-SEPARATED-GLOBNAMES) New in version 2.0.
A list of wildcard environment variable names which shall be copied from the tox invocation environment to the test environment when executing test commands.  If a specified environment variable doesn't exist in the tox invocation environment it is ignored. You can use * and ? to match multiple environment variables with one name.
Some variables are always passed through to ensure the basic functionality of standard library functions or tooling like pip:
• passed through on all platforms: PATH, LANG, LANGUAGE, LD_LIBRARY_PATH, PIP_INDEX_URL
Windows: SYSTEMDRIVE, SYSTEMROOT, PATHEXT, TEMP, TMP NUMBER_OF_PROCESSORS, USERPROFILE, MSYSTEM
• Others (e.g. UNIX, macOS): TMPDIR
You can override these variables with the setenv option.
If defined the TOX_TESTENV_PASSENV environment variable (in the tox invocation environment) can define additional space-separated variable names that are to be passed down to the test command environment.
Changed in version 2.7: PYTHONPATH will be passed down if explicitly defined. If PYTHONPATH exists in the host environment but is not declared in passenv a warning will be emitted.
recreate=false(true|false) Always recreate virtual environment if this option is true.
downloadcache(PATH) IGNORED -- Since pip-8 has caching by default this option is now ignored.  Please remove it from your configs as a future tox version might bark on it.
sitepackages=false(true|false) Set to true if you want to create virtual environments that also have access to globally installed packages.
WARNING: In cases where a command line tool is also installed globally you have to make sure that you use the tool installed in the virtualenv by using python -m <command line tool> (if supported by the tool) or {envbindir}/<command line tool>.
If you forget to do that you will get a warning like this:
WARNING: test command found but not installed in testenv cmd: /path/to/parent/interpreter/bin/<some command> env: /foo/bar/.tox/python Maybe you forgot to specify a dependency? See also the whitelist_externals envconfig setting.
alwayscopy=false(true|false) Set to true if you want virtualenv to always copy files rather than symlinking.
This is useful for situations where hardlinks don't work (e.g. running in VMS with Windows guests).
args_are_paths=false(true|false) Treat positional arguments passed to tox as file system paths and - if they exist on the filesystem - rewrite them according to the changedir. Default is true due to the exists-on-filesystem check it's usually safe to try rewriting.
envtmpdir={envdir}/tmp(PATH) Defines a temporary directory for the virtualenv which will be cleared each time before the group of test commands is invoked.
envlogdir={envdir}/log(PATH) Defines a directory for logging where tox will put logs of tool invocation.
indexserver(URL) New in version 0.9.
(DEPRECATED, will be removed in a future version) Multi-line name = URL definitions of python package servers.  Dependencies can specify using a specified index server through the :indexservername:depname pattern.  The default indexserver definition determines where unscoped dependencies and the sdist install installs from.  Example:
[tox] indexserver = default = https://mypypi.org
will make tox install all dependencies from this PyPI index server (including when installing the project sdist package).
envdir={toxworkdir}/{envname}(PATH) New in version 1.5.
User can set specific path for environment. If path would not be absolute it would be treated as relative to {toxinidir}.
usedevelop=false(true|false) New in version 1.6.
Install the current package in development mode with "setup.py develop" instead of installing from the sdist package. (This uses pip's -e option, so should be avoided if you've specified a custom install_command that does not support -e).
skip_install=false(true|false) New in version 1.9.
Do not install the current package. This can be used when you need the virtualenv management but do not want to install the current package into that environment.
ignore_outcome=false(true|false) New in version 2.2.
If set to true a failing result of this testenv will not make tox fail, only a warning will be produced.
extras(MULTI-LINE-LIST) New in version 2.4.
A list of "extras" to be installed with the sdist or develop install.  For example, extras = testing is equivalent to [testing] in a pip install command. These are not installed if skip_install is true.
description=no description(SINGLE-LINE-TEXT) A short description of the environment, this will be used to explain the environment to the user upon listing environments for the command line with any level of verbosity higher than zero.
Substitutions Any key=value setting in an ini-file can make use of value substitution through the {...} string-substitution pattern.
You can escape curly braces with the \ character if you need them, for example:
commands = echo "\{posargs\}" = {posargs}
Note some substitutions (e.g. posargs, env) may have addition values attached to it, via the : character (e.g. posargs - default value, env - key).  Such substitutions cannot have a space after the : character (e.g. {posargs: magic} while being at the start of a line inside the ini configuration (this would be parsed as factorial {posargs, having value magic).
Globally available substitutions {toxinidir} the directory where tox.ini is located
{toxworkdir} the directory where virtual environments are created and sub directories for packaging reside.
{homedir} the user-home directory path.
{distdir} the directory where sdist-packages will be created in
{distshare} (DEPRECATED) the directory where sdist-packages will be copied to so that they may be accessed by other processes or tox runs.
{:}    OS-specific path separator (: os *nix family, ; on Windows). May be used in setenv, when target variable is path variable (e.g. PATH or PYTHONPATH).
substitutions for virtualenv-related sections {envname} the name of the virtual environment
{envpython} path to the virtual Python interpreter
{envdir} directory of the virtualenv hierarchy
{envbindir} directory where executables are located
{envsitepackagesdir} directory where packages are installed.  Note that architecture-specific files may appear in a different directory.
{envtmpdir} the environment temporary directory
{envlogdir} the environment log directory
environment variable substitutions If you specify a substitution string like this:
{env:KEY}
then the value will be retrieved as os.environ['KEY'] and raise an Error if the environment variable does not exist.
environment variable substitutions with default values If you specify a substitution string like this:
{env:KEY:DEFAULTVALUE}
then the value will be retrieved as os.environ['KEY'] and replace with DEFAULTVALUE if the environment variable does not exist.
If you specify a substitution string like this:
{env:KEY:}
then the value will be retrieved as os.environ['KEY'] and replace with an empty string if the environment variable does not exist.
Substitutions can also be nested. In that case they are expanded starting from the innermost expression:
{env:KEY:{env:DEFAULT_OF_KEY}}
the above example is roughly equivalent to os.environ.get('KEY', os.environ['DEFAULT_OF_KEY'])
interactive shell substitution It's possible to inject a config value only when tox is running in interactive shell (standard input): {tty:ON_VALUE:OFF_VALUE}
The first value is the value to inject when the interactive terminal is available, the second value is the value to use when it's not. The later on is optional. A good use case for this is e.g. passing in the --pdb flag for pytest.
substitutions for positional arguments in commands New in version 1.0.
{posargs:DEFAULTS}
then the value will be replaced with positional arguments as provided to the tox command:
tox arg1 arg2
In this instance, the positional argument portion will be replaced with arg1 arg2. If no positional arguments were specified, the value of DEFAULTS will be used instead. If DEFAULTS contains other substitution strings, such as {env:*}, they will be interpreted.,
Use a double -- if you also want to pass options to an underlying test command, for example:
tox -- --opt1 ARG1
will make the --opt1 ARG1 appear in all test commands where [] or {posargs} was specified.  By default (see args_are_paths setting), tox rewrites each positional argument if it is a relative path and exists on the filesystem to become a path relative to the changedir setting.
Previous versions of tox supported the [.*] pattern to denote positional arguments with defaults. This format has been deprecated.  Use {posargs:DEFAULTS} to specify those.
Substitution for values from other sections New in version 1.4.
Values from other sections can be referred to via:
{[sectionname]valuename}
which you can use to avoid repetition of config values.  You can put default values in one section and reference them in others to avoid repeating the same values:
[base] deps = pytest mock pytest-xdist
[testenv:dulwich] deps = dulwich {[base]deps}
[testenv:mercurial] deps = mercurial {[base]deps}
Generating environments, conditional settings New in version 1.8.
Suppose you want to test your package against python2.7, python3.6 and against several versions of a dependency, say Django 1.5 and Django 1.6. You can accomplish that by writing down 2*2 = 4 [testenv:*] sections and then listing all of them in envlist.
However, a better approach looks like this:
[tox] envlist = {py27,py36}-django{15,16}
[testenv] deps = pytest django15: Django>=1.5,<1.6 django16: Django>=1.6,<1.7 py36: unittest2 commands = pytest
This uses two new facilities of tox-1.8:
• generative envlist declarations where each envname consists of environment parts or "factors"
• "factor" specific settings
Let's go through this step by step.
Generative envlist envlist = {py36,py27}-django{15,16}
This is bash-style syntax and will create 2*2=4 environment names like this:
py27-django15 py27-django16 py36-django15 py36-django16
You can still list environments explicitly along with generated ones:
envlist = {py27,py36}-django{15,16}, docs, flake
Keep in mind that whitespace characters (except newline) within {} are stripped, so the following line defines the same environment names:
envlist = {py27,py36}-django{ 15, 16 }, docs, flake
NOTE: To help with understanding how the variants will produce section values, you can ask tox to show their expansion with a new option:
$ tox -l py27-django15 py27-django16 py36-django15 py36-django16 docs flake
Factors and factor-conditional settings Parts of an environment name delimited by hyphens are called factors and can be used to set values conditionally. In list settings such as deps or commands you can freely intermix optional lines with unconditional ones:
[testenv] deps = pytest django15: Django>=1.5,<1.6 django16: Django>=1.6,<1.7 py36: unittest2
Reading it line by line:
• pytest will be included unconditionally,
• Django>=1.5,<1.6 will be included for environments containing django15 factor,
• Django>=1.6,<1.7 similarly depends on django16 factor,
• unittest will be loaded for Python 3.6 environments.
tox provides a number of default factors corresponding to Python interpreter versions. The conditional setting above will lead to either python3.6 or python2.7 used as base python, e.g. python3.6 is selected if current environment contains py36 factor.
NOTE: Configuring basepython for environments using default factors will result in a warning. Configure ignore_basepython_conflict if you wish to explicitly ignore these conflicts, allowing you to define a global basepython for all environments except those with default factors.
Complex factor conditions Sometimes you need to specify the same line for several factors or create a special case for a combination of factors. Here is how you do it:
[testenv] deps = py34-mysql: PyMySQL     ; use if both py34 and mysql are in the env name py27,py36: urllib3      ; use if either py36 or py27 are in the env name py{27,36}-sqlite: mock  ; mocking sqlite in python 2.x & 3.6 !py34-sqlite: mock      ; mocking sqlite, except in python 3.4 sqlite-!py34: mock      ; (same as the line above)
Take a look at the first deps line. It shows how you can special case something for a combination of factors, by just hyphenating the combining factors together. This particular line states that PyMySQL will be loaded for python 3.4, mysql environments, e.g. py34-django15-mysql and py34-django16-mysql.
The second line shows how you use the same setting for several factors - by listing them delimited by commas. It's possible to list not only simple factors, but also their combinations like py27-sqlite,py36-sqlite.
The remaining lines all have the same effect and use conditions equivalent to py27-sqlite,py36-sqlite. They have all been added only to help demonstrate the following:
• how factor expressions get expanded the same way as in envlist
• how to use negated factor conditions by prefixing negated factors with !
• that the order in which factors are hyphenated together does not matter
NOTE: Factors don't do substring matching against env name, instead every hyphenated expression is split by - and if ALL of its non-negated factors and NONE of its negated ones are also factors of an env then that condition is considered to hold for that env.
For example, environment py36-mysql-!dev:
• would be matched by expressions py36, py36-mysql or mysql-py36,
• but not py2, py36-sql or py36-mysql-dev.
Factors and values substitution are compatible It is possible to mix both values substitution and factor expressions. For example:
[tox] envlist = py27,py36,coverage
[testenv] deps = flake8 coverage: coverage
[testenv:py27] deps = {[testenv]deps} pytest
With the previous configuration, it will install:
• flake8 and pytest packages for py27 environment.
• flake8 package for py36 environment.
• flake8 and coverage packages for coverage environment.
Advanced settings Handle interpreter directives with long lengths For systems supporting executable text files (scripts with a shebang), the system will attempt to parse the interpreter directive to determine the program to execute on the target text file. When tox prepares a virtual environment in a file container which has a large length (e.x. using Jenkins Pipelines), the system might not be able to invoke shebang scripts which define interpreters beyond system limits (e.x. Linux as a limit of 128; BINPRM_BUF_SIZE). To workaround an environment which suffers from an interpreter directive limit, a user can bypass the system's interpreter parser by defining the TOX_LIMITED_SHEBANG environment variable before invoking tox:
export TOX_LIMITED_SHEBANG=1
When the workaround is enabled, all tox-invoked text file executables will have their interpreter directive parsed by and explicitly executed by tox.
Injected environment variables tox will inject the following environment variables that you can use to test that your command is running within tox:
New in version 3.4.
• TOX_WORK_DIR env var is set to the tox work directory
• TOX_ENV_NAME is set to the current running tox environment name
• TOX_ENV_DIR is set to the current tox environments working dir.
• TOX_PACKAGE the packaging phases outcome path (useful to inspect and make assertion of the built package itself).
note   this applies for all tox envs (isolated packaging too) and all external commands called (e.g. install command - pip).
Other Rules and notes • path specifications: if a specified path is a relative path it will be considered as relative to the toxinidir, the directory where the configuration file resides.
cli tox tox options
usage: tox [--version] [-h] [--help-ini] [-v] [-q] [--showconfig] [-l] [-a] [-c CONFIGFILE] [-e envlist] [--notest] [--sdistonly] [-p VAL] [-o] [--parallel--safe-build] [--installpkg PATH] [--develop] [-i URL] [--pre] [-r] [--result-json PATH] [--hashseed SEED] [--force-dep REQ] [--sitepackages] [--alwayscopy] [-s [val]] [--workdir PATH] [args [args ...]]
args   additional arguments available to command positional substitution
--version report version information to stdout.
-h, --help show help about options
--help-ini, --hi show help about ini-names
-v     increase verbosity of reporting output.-vv mode turns off output redirection for package installation, above level two verbosity flags are passed through to pip (with two less level)
-q     progressively silence reporting output.
--showconfig show configuration information for all environments.
-l, --listenvs show list of test environments (with description if verbose)
-a, --listenvs-all show list of all defined environments (with description if verbose)
-c <configfile> config file name or directory with 'tox.ini' file.
-e <envlist> work against specified environments (ALL selects all).
--notest skip invoking test commands.
--sdistonly only perform the sdist packaging activity.
-p <val>, --parallel <val> run tox environments in parallel, the argument controls limit: all, auto - cpu count, some positive number, zero is turn off
-o, --parallel-live connect to stdout while running environments
--parallel--safe-build (deprecated) ensure two tox builds can run in parallel (uses a lock file in the tox workdir with .lock extension)
--installpkg <path> use specified package for installation into venv, instead of creating an sdist.
--develop install package in the venv using 'setup.py develop' via 'pip -e .'
-i <url>, --index-url <url> set indexserver url (if URL is of form name=url set the url for the 'name' indexserver, specifically)
--pre  install pre-releases and development versions of dependencies. This will pass the --pre option to install_command (pip by default).
-r, --recreate force recreation of virtual environments
--result-json <path> write a json file with detailed information about all commands and results involved.
--hashseed <seed> set PYTHONHASHSEED to SEED before running commands.  Defaults to a random integer in the range [1, 4294967295] ([1, 1024] on Windows). Passing 'noset' suppresses this behavior.
--force-dep <req> Forces a certain version of one of the dependencies when configuring the virtual environment. REQ Examples 'pytest<2.7' or 'django>=1.6'.
--sitepackages override sitepackages setting to True in all envs
--alwayscopy override alwayscopy setting to True in all envs
-s, --skip-missing-interpreters don't fail tests for missing interpreters: {config,true,false} choice
--workdir <path> tox working directory
support and contact channels Getting in contact:
• join the tox-dev mailing list for tox related questions and development discussions.
• file a report on the issue tracker
• hang out on the irc.freenode.net #pylib channel
• fork the github repository and submit merge/pull requests (see the developers help page -- developers)
paid professional support contact holger at merlinux.eu, an association of experienced well-known Python developers.
Changelog history Versions follow Semantic Versioning (<major>.<minor>.<patch>).  Backward incompatible (breaking) changes will only be introduced in major versions with advance notice in the Deprecations section of releases.
v3.7.0 (2019-01-11) Features • Parallel mode added (alternative to detox which is being deprecated), for more details see parallel_mode - by @gaborbernat.  #439
• Added command line shortcut -s for --skip-missing-interpreters - by @evandrocoan #1119
Deprecations (removal in next major release) • Whitelisting of externals will be mandatory in tox 4: issue a deprecation warning as part of the already existing warning - by @obestwalter #1129
Documentation • Clarify explanations in examples and avoid unsupported end line comments - by @obestwalter #1110
• Set to PULL_REQUEST_TEMPLATE.md use relative instead of absolute URLs - by @evandrocoan Fixed PULL_REQUEST_TEMPLATE.md path for changelog/examples.rst to docs/changelog/examples.rst - by @evandrocoan #1120
v3.6.1 (2018-12-24) Features • if the packaging phase successfully builds a package set it as environment variable under TOX_PACKAGE (useful to make assertions on the built package itself, instead of just how it ends up after installation) - by @gaborbernat (#1081)
v3.6.0 (2018-12-13) Bugfixes • On windows, check sys.executable before others for interpreter version lookup.  This matches what happens on non-windows. (#1087)
• Don't rewrite {posargs} substitution for absolute paths. (#1095)
• Correctly fail tox --notest when setup fails. (#1097)
Documentation • Update Contributor Covenant URL to use https:// - by @jdufresne. (‐ #1082)
• Correct the capitalization of PyPI throughout the documentation - by @jdufresne. (#1084)
• Link to related projects (Invoke and Nox) from the documentation - by @theacodes. (#1088)
Miscellaneous • Include the license file in the wheel distribution - by @jdufresne. (‐ #1083)
v3.5.3 (2018-10-28) Bugfixes • Fix bug with incorrectly defactorized dependencies - by @bartsanchez (‐ #706)
• do the same transformation to egg_info folders that pkg_resources does; this makes it possible for hyphenated names to use the develop-inst-noop optimization (cf. 910), which previously only worked with non-hyphenated egg names - by @hashbrowncipher (#1051)