Repository: tox-dev/tox
Pull request #934 (2018-08-09 10:15:18)

Allow to run multiple tox instances in parallel #849
At the moment the build phase is not thread safe. Running multiple
instances of tox will highly likely cause one of the instances
to fail as the two processes will step on each others toe while
creating the sdist package. This is especially annoying in CI
environments where you want to run tox targets in parallle (e..g
Jenkins).

By passing in ``--parallel--safe-build`` flag tox automatically
generates a unique dist folder for the current build. This way
each build can use it's own version built package (in the install
phase after the build), and we avoid the need to lock while
building. Once the tox session finishes remove such build folders
to avoid ever expanding source trees when using this feature.


codecov-io: # [Codecov](https://codecov.io/gh/tox-dev/tox/pull/934?src=pr&el=h1) Report
> Merging [#934](https://codecov.io/gh/tox-dev/tox/pull/934?src=pr&el=desc) into [master](https://codecov.io/gh/tox-dev/tox/commit/e557705755848139560893480a1975c24f4f9a5a?src=pr&el=desc) will **increase** coverage by `0.17%`.
> The diff coverage is `100%`.

[![Impacted file tree graph](https://codecov.io/gh/tox-dev/tox/pull/934/graphs/tree.svg?width=650&height=150&src=pr&token=DYodAwDCZ5)](https://codecov.io/gh/tox-dev/tox/pull/934?src=pr&el=tree)

```diff
@@            Coverage Diff            @@
##           master    #934      +/-   ##
=========================================
+ Coverage   92.63%   92.8%   +0.17%     
=========================================
  Files          12      12              
  Lines        2335    2363      +28     
  Branches      409     411       +2     
=========================================
+ Hits         2163    2193      +30     
+ Misses        109     108       -1     
+ Partials       63      62       -1
```


| [Impacted Files](https://codecov.io/gh/tox-dev/tox/pull/934?src=pr&el=tree) | Coverage Δ | |
|---|---|---|
| [src/tox/\_pytestplugin.py](https://codecov.io/gh/tox-dev/tox/pull/934/diff?src=pr&el=tree#diff-c3JjL3RveC9fcHl0ZXN0cGx1Z2luLnB5) | `92.65% <100%> (+0.68%)` | :arrow_up: |
| [src/tox/config.py](https://codecov.io/gh/tox-dev/tox/pull/934/diff?src=pr&el=tree#diff-c3JjL3RveC9jb25maWcucHk=) | `95.93% <100%> (+0.02%)` | :arrow_up: |
| [src/tox/session.py](https://codecov.io/gh/tox-dev/tox/pull/934/diff?src=pr&el=tree#diff-c3JjL3RveC9zZXNzaW9uLnB5) | `91.97% <100%> (+0.39%)` | :arrow_up: |

------

[Continue to review full report at Codecov](https://codecov.io/gh/tox-dev/tox/pull/934?src=pr&el=continue).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/tox-dev/tox/pull/934?src=pr&el=footer). Last update [e557705...55a3c0d](https://codecov.io/gh/tox-dev/tox/pull/934?src=pr&el=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments).


Patch:
diff --git a/changelog/849.feature.rst b/changelog/849.feature.rst
new file mode 100644
index 00000000..5f0daf69
--- /dev/null
+++ b/changelog/849.feature.rst
@@ -0,0 +1,2 @@
+Allow to run multiple tox instances in parallel by providing the
+```--parallel--safe-build`` flag. - by :user:`gaborbernat`
diff --git a/doc/example/jenkins.rst b/doc/example/jenkins.rst
index 9d8a96d4..8c88ad4c 100644
--- a/doc/example/jenkins.rst
+++ b/doc/example/jenkins.rst
@@ -188,4 +188,27 @@ Linux as a limit of 128). There are two methods to workaround this issue:
     :ref:`long interpreter directives` for more information).
 
 
+Running tox environments in parallel
+------------------------------------
+
+Jenkins has parallel stages allowing you to run commands in parallel, however tox package
+building it is not parallel safe. Use the ``--parallel--safe-build`` flag to enable parallel safe
+builds. Here's a generic stage definition demonstrating this:
+
+.. code-block:: groovy
+
+    stage('run tox envs') {
+      steps {
+        script {
+          def envs = sh(returnStdout: true, script: "tox -l").trim().split('\n')
+          def cmds = envs.collectEntries({ tox_env ->
+            [tox_env, {
+              sh "tox --parallel--safe-build -vve $tox_env"
+            }]
+          })
+          parallel(cmds)
+        }
+      }
+    }
+
 .. include:: ../links.rst
diff --git a/src/tox/config.py b/src/tox/config.py
index d3e6e764..43c80d39 100755
--- a/src/tox/config.py
+++ b/src/tox/config.py
@@ -8,6 +8,7 @@ import re
 import shlex
 import string
 import sys
+import uuid
 import warnings
 from fnmatch import fnmatchcase
 from subprocess import list2cmdline
@@ -390,6 +391,12 @@ def tox_addoption(parser):
         dest="sdistonly",
         help="only perform the sdist packaging activity.",
     )
+    parser.add_argument(
+        "--parallel--safe-build",
+        action="store_true",
+        dest="parallel_safe_build",
+        help="ensure two tox builds can run in parallel",
+    )
     parser.add_argument(
         "--installpkg",
         action="store",
@@ -864,7 +871,7 @@ def make_hashseed():
 
 
 class parseini:
-    def __init__(self, config, inipath):
+    def __init__(self, config, inipath):  # noqa
         config.toxinipath = inipath
         config.toxinidir = config.toxinipath.dirpath()
 
@@ -950,6 +957,10 @@ class parseini:
 
         reader.addsubstitutions(toxworkdir=config.toxworkdir)
         config.distdir = reader.getpath("distdir", "{toxworkdir}/dist")
+        if config.option.parallel_safe_build:
+            config.distdir = py.path.local(config.distdir.dirname).join(
+                "{}-{}".format(config.distdir.basename, str(uuid.uuid4()))
+            )
         reader.addsubstitutions(distdir=config.distdir)
         config.distshare = reader.getpath("distshare", distshare_default)
         reader.addsubstitutions(distshare=config.distshare)
diff --git a/src/tox/session.py b/src/tox/session.py
index 8ff104af..19507655 100644
--- a/src/tox/session.py
+++ b/src/tox/session.py
@@ -42,7 +42,11 @@ def cmdline(args=None):
 def main(args):
     try:
         config = prepare(args)
-        retcode = Session(config).runcommand()
+        try:
+            retcode = Session(config).runcommand()
+        finally:
+            if config.option.parallel_safe_build:
+                config.distdir.remove(ignore_errors=True)
         if retcode is None:
             retcode = 0
         raise SystemExit(retcode)


Test patch:
diff --git a/src/tox/_pytestplugin.py b/src/tox/_pytestplugin.py
index 7602a481..564d0505 100644
--- a/src/tox/_pytestplugin.py
+++ b/src/tox/_pytestplugin.py
@@ -1,6 +1,7 @@
 from __future__ import print_function, unicode_literals
 
 import os
+import sys
 import textwrap
 import time
 from fnmatch import fnmatch
@@ -10,10 +11,11 @@ import pytest
 import six
 
 import tox
+from tox import venv
 from tox.config import parseconfig
 from tox.result import ResultLog
 from tox.session import Session, main
-from tox.venv import VirtualEnv
+from tox.venv import CreationConfig, VirtualEnv, getdigest
 
 mark_dont_run_on_windows = pytest.mark.skipif(os.name == "nt", reason="non windows test")
 mark_dont_run_on_posix = pytest.mark.skipif(os.name == "posix", reason="non posix test")
@@ -70,7 +72,16 @@ def cmd(request, capfd, monkeypatch):
         key = str(b"PYTHONPATH")
         python_paths = (i for i in (str(os.getcwd()), os.getenv(key)) if i)
         monkeypatch.setenv(key, os.pathsep.join(python_paths))
+
         with RunResult(capfd, argv) as result:
+            prev_run_command = Session.runcommand
+
+            def run_command(self):
+                result.session = self
+                return prev_run_command(self)
+
+            monkeypatch.setattr(Session, "runcommand", run_command)
+
             try:
                 main([str(x) for x in argv])
                 assert False  # this should always exist with SystemExit
@@ -91,6 +102,7 @@ class RunResult:
         self.duration = None
         self.out = None
         self.err = None
+        self.session = None
 
     def __enter__(self):
         self._start = time.time()
@@ -373,3 +385,38 @@ def create_files(base, filedefs):
         elif isinstance(value, six.string_types):
             s = textwrap.dedent(value)
             base.join(key).write(s)
+
+
+@pytest.fixture()
+def mock_venv(monkeypatch):
+    """This creates a mock virtual environment (e.g. will inherit the current interpreter).
+    Note: because we inherit, to keep things sane you must call the py environment and only that;
+    and cannot install any packages. """
+
+    class ProxyCurrentPython:
+        @classmethod
+        def readconfig(cls, path):
+            assert path.dirname.endswith("{}py".format(os.sep))
+            return CreationConfig(
+                md5=getdigest(sys.executable),
+                python=sys.executable,
+                version=tox.__version__,
+                sitepackages=False,
+                usedevelop=False,
+                deps=[],
+                alwayscopy=False,
+            )
+
+    monkeypatch.setattr(CreationConfig, "readconfig", ProxyCurrentPython.readconfig)
+
+    def venv_lookup(venv, name):
+        assert name == "python"
+        return sys.executable
+
+    monkeypatch.setattr(VirtualEnv, "_venv_lookup", venv_lookup)
+
+    @tox.hookimpl
+    def tox_runenvreport(venv, action):
+        return []
+
+    monkeypatch.setattr(venv, "tox_runenvreport", tox_runenvreport)
diff --git a/tests/test_session.py b/tests/test_session.py
index 010ea141..a4366d56 100644
--- a/tests/test_session.py
+++ b/tests/test_session.py
@@ -1,4 +1,5 @@
 import re
+import uuid
 
 import pytest
 
@@ -76,3 +77,23 @@ def test_minversion(cmd, initproj):
         r"ERROR: MinVersionError: tox version is .*," r" required is at least 6.0", result.out
     )
     assert result.ret
+
+
+def test_tox_parallel_build_safe(initproj, cmd, mock_venv):
+    initproj(
+        "env_var_test",
+        filedefs={
+            "tox.ini": """
+                          [tox]
+                          envlist = py
+                          [testenv]
+                          skip_install = true
+                          commands = python --version
+                      """
+        },
+    )
+    result = cmd("--parallel--safe-build")
+    basename = result.session.config.distdir.basename
+    assert basename.startswith("dist-")
+    assert uuid.UUID(basename[len("dist-") :], version=4)
+    assert not result.session.config.distdir.exists()