Repository: pypa/build
Pull request #153 (2020-10-21 17:49:20)

tox: don't run integration tests by default
People can opt in via posargs.

Signed-off-by: Filipe Laíns <lains@archlinux.org>

FFY00: They can't, we specifically did not add that to keep things simple. We have two options, `--run-integration`, that enables the integration tests, and `--only-integration`, that enables the integration tests and disables the rest.

When we discussed moving the integration tests to pytest we agreed that they would be disabled by default, most of the time you don't want to run them. They will still be run on the CI.
pganssle: > I prefer to run them by default, and people can opt-out via the posargs.

Same here. I have not found them to be particularly slow. When I run `tox` unqualified I want it to run all the tests so I don't have to wait for CI to provision a million environments and whatnot. When I'm testing in a tight loop I use posargs to skip certain tests.

If the tests took an absurdly long amount of time like multiple minutes I might feel differently, but my understanding was that the integration tests were skipped to avoid requiring hitting the internet, which is not a concern 95% of the time.

Perhaps we can compromise and allow flagging this with an environment variable, so users can easily set it for an entire shell session (or perhaps flip the default by including it in `~/.bashrc`)?
gaborbernat: > When we discussed moving the integration tests to pytest we agreed that they would be disabled by default, most of the time you don't want to run them. They will still be run on the CI.

I think we should reconsider. I'd keep them on always. Remove ``--only-integration`` because is useless. And only run them when ``--run-integration`` is passed, and this is passed by default via posarg inside tox. Users can easily then skip it by doing: ``tox -e py39 -- -v`` (this replaces the posarg of ``--run-integration`` with just ``-v``).
FFY00: I don't know how long it takes there, but here it takes ~1m15, which compared to the 8s of the test run without them, I think it's an absurd amount of time, but that's just my opinion. To me, 1min is not a reasonable delay in the development loop, at least when it can be avoided.

The integration tests were not designed to be run in the normal suite, that's why they build several different projects in different ways. I think they are something definitely good to keep around as a sanity check, but they are generally redundant with the rest of the tests, and with themselves. There are some exceptions, but they are *very* minor. These tests are not something I want to run all the time. I do not want to add over 1min to the cycle to run redundant tests.

Running the integration tests by default is also, IMO, a bad UX for contributors. They will likely not be affected by them, it will just be something they have to wait on. Sure they can disable them, but this is not clear to new contributors. I find the current approach much better, the tests will be disabled and pytest will report the reason. Users will be notified that the tests were skipped and if want to run them they can easily find out how.

```
====================================================================================================================================== short test summary info =======================================================================================================================================
SKIPPED [2] tests/test_integration.py: integration tests not run (no --run-integration flag)
SKIPPED [60] tests/test_integration.py:70: integration tests not run (no --run-integration flag)
```

Because of this, I am -1 on running them by default.

> Perhaps we can compromise and allow flagging this with an environment variable, so users can easily set it for an entire shell session (or perhaps flip the default by including it in `~/.bashrc`)?

I can accept this as a compromise. I will not be the target user for it, but if it makes your life easier, sure.

> Remove `--only-integration` because is useless.

It isn't.

> Users can easily then skip it by doing: `tox -e py39 -- -v` (this replaces the posarg of `--run-integration` with just `-v`). This way less to type.

This is definitely not intuitive.
gaborbernat: Intuitiveness is relative. For me what we have is not intuitive either 🤷‍♂️

Patch:
diff --git a/.github/workflows/check.yml b/.github/workflows/check.yml
index 15e93f1..5eb3704 100644
--- a/.github/workflows/check.yml
+++ b/.github/workflows/check.yml
@@ -78,13 +78,13 @@ jobs:
         run: tox -e ${{env.BASE}} --skip-pkg-install
 
       - name: Run test suite via path
-        run: tox -e ${{env.BASE}}-path --skip-pkg-install
+        run: tox -e ${{env.BASE}}-path --skip-pkg-install -- --run-integration
 
       - name: Run test suite via sdist
-        run: tox -e ${{env.BASE}}-sdist --skip-pkg-install
+        run: tox -e ${{env.BASE}}-sdist --skip-pkg-install -- --run-integration
 
       - name: Run test suite via wheel
-        run: tox -e ${{env.BASE}}-wheel --skip-pkg-install
+        run: tox -e ${{env.BASE}}-wheel --skip-pkg-install -- --run-integration
 
       - name: Rename coverage report file
         run: |
diff --git a/setup.cfg b/setup.cfg
index 7a4c89c..02cfdb2 100644
--- a/setup.cfg
+++ b/setup.cfg
@@ -26,7 +26,7 @@ project_urls =
 packages = find:
 install_requires =
     packaging
-    pep517
+    pep517>=0.9.1
     toml
     importlib-metadata;python_version < "3.8"
     typing;python_version<"3"
@@ -50,6 +50,9 @@ test =
     pytest
     pytest-cov
     pytest-mock
+typing =
+    mypy==0.790
+    typing-extensions>=3.7.4.3
 
 [options.packages.find]
 where = src
@@ -78,6 +81,11 @@ omit =
     *bin/pyproject-build
     *bin\pyproject-build.exe
 
+[coverage:report]
+exclude_lines =
+    \#\s*pragma: no cover
+    ^\s*raise NotImplementedError\b
+
 [coverage:paths]
 source =
     src
diff --git a/src/build/__init__.py b/src/build/__init__.py
index 885a926..d6d3455 100644
--- a/src/build/__init__.py
+++ b/src/build/__init__.py
@@ -10,14 +10,12 @@ import difflib
 import os
 import sys
 import warnings
-
 from typing import Dict, Iterator, List, Mapping, Optional, Set, Union
 
 import pep517.wrappers
 import toml
 import toml.decoder
 
-
 if sys.version_info < (3,):
     FileNotFoundError = IOError
     PermissionError = OSError
@@ -118,9 +116,14 @@ def _working_directory(path):  # type: (str) -> Iterator[None]
 
 
 class ProjectBuilder(object):
-    def __init__(self, srcdir='.', config_settings=None):  # type: (str, Optional[ConfigSettings]) -> None
+    def __init__(self, srcdir='.', config_settings=None, python_executable=sys.executable):
+        # type: (str, Optional[ConfigSettings], str) -> None
         """
-        :param srcdir: Source directory
+        Create a project builder.
+
+        :param srcdir: the source directory
+        :param config_settings: config settings for the build backend
+        :param python_executable: the python executable where the backend lives
         """
         self.srcdir = os.path.abspath(srcdir)
         self.config_settings = config_settings if config_settings else {}
@@ -152,7 +155,10 @@ class ProjectBuilder(object):
         self._backend = self._build_system['build-backend']
 
         self.hook = pep517.wrappers.Pep517HookCaller(
-            self.srcdir, self._backend, backend_path=self._build_system.get('backend-path')
+            self.srcdir,
+            self._backend,
+            backend_path=self._build_system.get('backend-path'),
+            python_executable=python_executable,
         )
 
     @property
diff --git a/src/build/__main__.py b/src/build/__main__.py
index 2d0aafe..a353f9d 100644
--- a/src/build/__main__.py
+++ b/src/build/__main__.py
@@ -9,8 +9,7 @@ import warnings
 from typing import List, Optional, TextIO, Type, Union
 
 from build import BuildBackendException, BuildException, ConfigSettings, ProjectBuilder
-from build.env import IsolatedEnvironment
-
+from build.env import IsolatedEnvBuilder
 
 __all__ = ['build', 'main', 'main_parser']
 
@@ -40,8 +39,10 @@ def _error(msg, code=1):  # type: (str, int) -> None  # pragma: no cover
     exit(code)
 
 
-def _build_in_isolated_env(builder, outdir, distributions):  # type: (ProjectBuilder, str, List[str]) -> None
-    with IsolatedEnvironment.for_current() as env:
+def _build_in_isolated_env(builder, outdir, distributions):
+    # type: (ProjectBuilder, str, List[str]) -> None
+    with IsolatedEnvBuilder() as env:
+        builder.hook.python_executable = env.executable
         env.install(builder.build_dependencies)
         for distribution in distributions:
             builder.build(distribution, outdir)
@@ -75,7 +76,6 @@ def build(srcdir, outdir, distributions, config_settings=None, isolation=True, s
 
     try:
         builder = ProjectBuilder(srcdir, config_settings)
-
         if isolation:
             _build_in_isolated_env(builder, outdir, distributions)
         else:
diff --git a/src/build/_compat.py b/src/build/_compat.py
new file mode 100644
index 0000000..3a5da79
--- /dev/null
+++ b/src/build/_compat.py
@@ -0,0 +1,42 @@
+import abc
+import sys
+from typing import Callable, TypeVar
+
+_T = TypeVar('_T')
+
+
+def add_metaclass(metaclass):  # type: (type) -> Callable[[_T], _T]
+    """Class decorator for creating a class with a metaclass (borrowed from six code)."""
+
+    def wrapper(cls):  # type: (_T) -> _T
+        orig_vars = cls.__dict__.copy()
+        slots = orig_vars.get('__slots__')
+        if slots is not None:
+            if isinstance(slots, str):  # pragma: no cover
+                slots = [slots]  # pragma: no cover
+            for slots_var in slots:  # pragma: no cover
+                orig_vars.pop(slots_var)  # pragma: no cover
+        orig_vars.pop('__dict__', None)
+        orig_vars.pop('__weakref__', None)
+        if hasattr(cls, '__qualname__'):
+            orig_vars['__qualname__'] = cls.__qualname__  # type: ignore
+        return metaclass(cls.__name__, cls.__bases__, orig_vars)  # type: ignore
+
+    return wrapper
+
+
+if sys.version_info[0] == 2:
+    abstractproperty = abc.abstractproperty
+else:
+    from typing import Any, cast
+
+    F = TypeVar('F', bound=Callable[..., Any])
+
+    def abstractproperty(func):  # type: (F) -> F
+        return cast(F, property(abc.abstractmethod(func)))
+
+
+__all__ = (
+    'abstractproperty',
+    'add_metaclass',
+)
diff --git a/src/build/env.py b/src/build/env.py
index 760957e..0c1be7e 100644
--- a/src/build/env.py
+++ b/src/build/env.py
@@ -1,3 +1,7 @@
+"""
+Creates and manages isolated build environments.
+"""
+import abc
 import os
 import platform
 import shutil
@@ -5,237 +9,207 @@ import subprocess
 import sys
 import sysconfig
 import tempfile
-import types
-import typing
-from typing import Dict, Iterable, List, Optional, Sequence, Type
+from types import TracebackType
+from typing import Iterable, Optional, Tuple, Type
 
-if sys.version_info[0] == 2:
-    FileExistsError = OSError
+from ._compat import abstractproperty, add_metaclass
 
 try:
     import pip
 except ImportError:  # pragma: no cover
     pip = None  # pragma: no cover
 
-_HAS_SYMLINK = None  # type: Optional[bool]
 
+@add_metaclass(abc.ABCMeta)
+class IsolatedEnv(object):
+    """Abstract base of isolated build environments, as required by the build project."""
 
-def _fs_supports_symlink():  # type: () -> bool
-    if not hasattr(os, 'symlink'):
-        return False
+    @abstractproperty
+    def executable(self):  # type: () -> str
+        """Return the executable of the isolated build environment."""
+        raise NotImplementedError
 
-    if sys.platform.startswith(('aix', 'darwin', 'freebsd', 'linux')):
-        return True
-    else:
-        with tempfile.NamedTemporaryFile(prefix='TmP') as tmp_file:
-            temp_dir = os.path.dirname(tmp_file.name)
-            dest = os.path.join(temp_dir, '{}-{}'.format(tmp_file.name, 'b'))
-            try:
-                os.symlink(tmp_file.name, dest)
-                return True
-            except (OSError, NotImplementedError):
-                return False
+    @abc.abstractmethod
+    def install(self, requirements):  # type: (Iterable[str]) -> None
+        """
+        Install PEP-508 requirements into the isolated build environment.
+
+        :param requirements: PEP-508 requirements
+        """
+        raise NotImplementedError
+
+
+class IsolatedEnvBuilder(object):
+    def __init__(self):  # type: () -> None
+        """Builder object for isolated environment."""
+        self._path = None  # type: Optional[str]
+
+    def __enter__(self):  # type: () -> IsolatedEnv
+        """
+        Creates an isolated build environment.
+
+        :return: the isolated build environment
+        """
+        self._path = tempfile.mkdtemp(prefix='build-env-')
+        try:
+            executable, pip_executable = _create_isolated_env(self._path)
+            return _IsolatedEnvVenvPip(path=self._path, python_executable=executable, pip_executable=pip_executable)
+        except Exception:  # cleanup folder if creation fails
+            self.__exit__(*sys.exc_info())
+            raise
 
+    def __exit__(self, exc_type, exc_val, exc_tb):
+        # type: (Optional[Type[BaseException]], Optional[BaseException], Optional[TracebackType]) -> None
+        """
+        Delete the created isolated build environment.
 
-def fs_supports_symlink():  # type: () -> bool
-    global _HAS_SYMLINK
-    if _HAS_SYMLINK is None:
-        _HAS_SYMLINK = _fs_supports_symlink()
-    return _HAS_SYMLINK
+        :param exc_type: the type of exception raised (if any)
+        :param exc_val: the value of exception raised (if any)
+        :param exc_tb: the traceback of exception raised (if any)
+        """
+        if self._path is not None and os.path.exists(self._path):  # in case the user already deleted skip remove
+            shutil.rmtree(self._path)
 
 
-class IsolatedEnvironment(object):
+class _IsolatedEnvVenvPip(IsolatedEnv):
     """
     Isolated build environment context manager
 
     Non-standard paths injected directly to sys.path still be passed to the environment.
     """
 
-    _MANIPULATE_PATHS = ('purelib', 'platlib')
-
-    def __init__(self, remove_paths, _executable=sys.executable):  # type: (Sequence[str], str) -> None
-        """
-        :param remove_paths: Import paths that should be removed from the environment
+    def __init__(self, path, python_executable, pip_executable):
+        # type: (str, str, str) -> None
         """
-        self._env = {}  # type: Dict[str, Optional[str]]
-        self._env_vars = {}  # type: Dict[str, str]
-        self._path = None  # type: Optional[str]
-        self._remove_paths = []  # type: List[str]
-        self._executable = _executable
-        self._old_executable = None  # type: Optional[str]
-        self._pip_executable = None  # type: Optional[str]
+        Define an isolated build environment.
 
-        # normalize paths so that we can compare them -- required on case insensitive systems
-        for path in remove_paths:
-            self._remove_paths.append(os.path.normcase(path))
+        :param path: the path where the environment exists
+        :param python_executable: the python executable within the environment
+        :param pip_executable: an executable that allows installing packages within the environment
+        """
+        self._path = path
+        self._pip_executable = pip_executable
+        self._python_executable = python_executable
 
     @property
     def path(self):  # type: () -> str
-        if not self._path:
-            raise RuntimeError("{} context environment hasn't been entered yet".format(self.__class__.__name__))
-
+        """:return: the location of the isolated build environment"""
         return self._path
 
     @property
     def executable(self):  # type: () -> str
-        return self._executable
+        """:return: the python executable of the isolated build environment"""
+        return self._python_executable
 
-    @classmethod
-    def for_current(cls):  # type: () -> IsolatedEnvironment
-        """
-        Creates an isolated environment for the current interpreter
+    def install(self, requirements):  # type: (Iterable[str]) -> None
         """
-        remove_paths = os.environ.get('PYTHONPATH', '').split(os.pathsep)
-        # also remove the pth file adding this project (when installed via --develop install)
-        for path in sys.path:
-            if os.path.exists(os.path.join(path, 'build.egg-info')):
-                remove_paths.append(path)  # pragma: no cover
+        Installs the specified PEP 508 requirements on the environment
 
-        return cls(remove_paths)
+        :param requirements: PEP-508 requirement specification to install
 
-    def _replace_env(self, key, new):  # type: (str, Optional[str]) -> None
-        """
-        Replaces an environment variable
+        :note: Passing non PEP 508 strings will result in undefined behavior, you *should not* rely on it. It is \
+               merely an implementation detail, it may change any time without warning.
         """
-        if not new:  # pragma: no cover
+        if not requirements:
             return
 
-        self._env[key] = os.environ.get(key, None)
-        os.environ[key] = new
+        with tempfile.NamedTemporaryFile('w+', prefix='build-reqs-', suffix='.txt', delete=False) as req_file:
+            req_file.write(os.linesep.join(requirements))
+        try:
+            cmd = [
+                self._pip_executable,
+                # on python2 if isolation is achieved via environment variables, we need to ignore those while calling
+                # host python (otherwise pip would not be available within it)
+                '-{}m'.format('E' if self._pip_executable == self.executable and sys.version_info[0] == 2 else ''),
+                'pip',
+                'install',
+                '--prefix',
+                self.path,
+                '--ignore-installed',
+                '--no-warn-script-location',
+                '-r',
+                os.path.abspath(req_file.name),
+            ]
+            subprocess.check_call(cmd)
+        finally:
+            os.unlink(req_file.name)
 
-    def _pop_env(self, key):  # type: (str) -> None
-        """
-        Removes an environment variable
-        """
-        self._env[key] = os.environ.pop(key, None)
 
-    def _restore_env(self):  # type: () -> None
-        """
-        Restores the initial environment variables
-        """
-        for key, val in self._env.items():
-            if val is None:
-                os.environ.pop(key, None)
-            else:
-                os.environ[key] = val
+if sys.version_info[0] == 2:  # noqa: C901 # disable if too complex
 
-    def _get_env_path(self, path):  # type: (str) -> Optional[str]
-        """
-        Returns sysconfig path from our environment
+    def _create_isolated_env(path):  # type: (str) -> Tuple[str, str]
         """
-        return sysconfig.get_path(path, vars=self._env_vars)
-
-    def _create_env_venv(self):  # type: () -> None
-        import venv
-
-        venv.EnvBuilder(with_pip=False).create(self.path)
+        On Python 2 we use the virtualenv package to provision a virtual environment.
 
-        env_scripts = self._get_env_path('scripts')
-        if not env_scripts:
-            raise RuntimeError("Couldn't get environment scripts path")
-        exe = 'pypy3' if platform.python_implementation() == 'PyPy' else 'python'
-        if os.name == 'nt':
-            exe = '{}.exe'.format(exe)
-
-        self._executable = os.path.join(self.path, env_scripts, exe)
-        if not os.path.exists(self._executable):
-            raise RuntimeError('Virtual environment creation failed, executable {} missing'.format(self._executable))
-
-        # Scenario 1: pip is available (either installed or via pth file) within the python executable alongside
-        # this projects environment: in this case we should be able to import it
-        if pip is not None:
-            self._pip_executable = sys.executable
-            return
-        # Scenario 2: this project is installed into a virtual environment that has no pip, but the matching system has
-        # Scenario 3: there's a pip executable on PATH
-        # Scenario 4: no pip can be found, we might be able to provision one into the isolated build env via ensurepip
-        cmd = [self.executable, '-Im', 'ensurepip', '--upgrade', '--default-pip']
-        try:
-            subprocess.check_call(cmd, cwd=self.path)
-        except subprocess.CalledProcessError:  # pragma: no cover
-            pass  # pragma: no cover
-        # avoid the setuptools from ensurepip to break the isolation
-        subprocess.check_call([self.executable, '-Im', 'pip', 'uninstall', 'setuptools', '-y'])
-        self._pip_executable = self.executable
-
-    def _create_env_virtualenv(self):  # type: () -> None
+        :param path: the folder where to create the isolated build environment
+        :return: the isolated build environment executable, and the pip to use to install packages into it
+        """
         from virtualenv import cli_run
 
-        cmd = [str(self.path), '--no-setuptools', '--no-wheel', '--activators', '']
+        cmd = [str(path), '--no-setuptools', '--no-wheel', '--activators', '']
         if pip is not None:
             cmd.append('--no-pip')
         result = cli_run(cmd, setup_logging=False)
-        self._executable = str(result.creator.exe)
-        self._pip_executable = self._executable
-        if pip is not None:
-            self._pip_executable = sys.executable
+        executable = str(result.creator.exe)
+        pip_executable = executable if pip is None else sys.executable
+        return executable, pip_executable
 
-    def _create_isolated_env(self):  # type: () -> None
-        if sys.version_info[0] == 2:
-            self._create_env_virtualenv()
 
-        else:
-            self._create_env_venv()
+else:
 
-    def __enter__(self):  # type: () -> IsolatedEnvironment
+    def _create_isolated_env(path):  # type: (str) -> Tuple[str, str]
         """
-        Set up the environment
+        On Python 3 we use the venv package from the standard library, and if host python has no pip the ensurepip
+        package to provision one into the created virtual environment.
 
-        The environment path should be empty
+        :param path: the folder where to create the isolated build environment
+        :return: the isolated build environment executable, and the pip to use to install packages into it
         """
-        self._path = tempfile.mkdtemp(prefix='build-env-')
-        self._env_vars = {
-            'base': self.path,
-            'platbase': self.path,
-        }
-
-        self._create_isolated_env()
+        import venv
 
-        self._pop_env('PIP_REQUIRE_VIRTUALENV')
-        # address https://github.com/pypa/pep517/pull/93
-        self._old_executable, sys.executable = sys.executable, self._executable
+        venv.EnvBuilder(with_pip=False).create(path)
+        executable = _find_executable(path)
 
-        return self
+        # Scenario 1: pip is available (either installed or via pth file) within the python executable alongside
+        # this projects environment: in this case we should be able to import it
+        if pip is not None:
+            pip_executable = sys.executable
+        else:
+            # Scenario 2: this project is installed into a virtual environment that has no pip, but the system has
+            # Scenario 3: there's a pip executable on PATH
+            # Scenario 4: no pip can be found, we might be able to provision one into the build env via ensurepip
+            cmd = [executable, '-Im', 'ensurepip', '--upgrade', '--default-pip']
+            try:
+                subprocess.check_call(cmd, cwd=path)
+            except subprocess.CalledProcessError:  # pragma: no cover
+                pass  # pragma: no cover
+            # avoid the setuptools from ensurepip to break the isolation
+            subprocess.check_call([executable, '-Im', 'pip', 'uninstall', 'setuptools', '-y'])
+            pip_executable = executable
+        return executable, pip_executable
 
-    def __exit__(self, typ, value, traceback):
-        # type: (Optional[Type[BaseException]], Optional[BaseException], Optional[types.TracebackType]) -> None
+    def _find_executable(path):  # type: (str) -> str
         """
-        Restores the everything to the original state
-        """
-        if self._old_executable is not None:
-            sys.executable = self._old_executable
-        if self.path and os.path.isdir(self.path):
-            shutil.rmtree(self.path)
-
-        self._restore_env()
+        Detect the executable within a virtual environment.
 
-    def install(self, requirements):  # type: (Iterable[str]) -> None
+        :param path: the location of the virtual environment
+        :return: the python executable
         """
-        Installs the specified PEP 508 requirements on the environment
+        config_vars = sysconfig.get_config_vars()
+        config_vars['base'] = path
+        env_scripts = sysconfig.get_path('scripts', vars=config_vars)
+        if not env_scripts:
+            raise RuntimeError("Couldn't get environment scripts path")
+        exe = 'pypy3' if platform.python_implementation() == 'PyPy' else 'python'
+        if os.name == 'nt':
+            exe = '{}.exe'.format(exe)
+        executable = os.path.join(path, env_scripts, exe)
+        if not os.path.exists(executable):
+            raise RuntimeError('Virtual environment creation failed, executable {} missing'.format(executable))
+        return executable
 
-        Passing non PEP 508 strings will result in undefined behavior, you
-        *should not* rely on it. It is merely an implementation detail, it may
-        change any time without warning.
-        """
-        if not requirements:
-            return
 
-        with tempfile.NamedTemporaryFile('w+', prefix='build-reqs-', suffix='.txt', delete=False) as req_file:
-            req_file.write(os.linesep.join(requirements))
-            req_file.close()
-            cmd = [
-                typing.cast(str, self._pip_executable),
-                # on python2 if isolation is achieved via environment variables, we need to ignore those while calling
-                # host python (otherwise pip would not be available within it)
-                '-{}m'.format('E' if self._pip_executable == self._executable and sys.version_info[0] == 2 else ''),
-                'pip',
-                'install',
-                '--prefix',
-                self.path,
-                '--ignore-installed',
-                '--no-warn-script-location',
-                '-r',
-                os.path.abspath(req_file.name),
-            ]
-            subprocess.check_call(cmd)
-            os.unlink(req_file.name)
+__all__ = (
+    'IsolatedEnvBuilder',
+    'IsolatedEnv',
+)
diff --git a/tox.ini b/tox.ini
index e32bb0d..91a3d4b 100644
--- a/tox.ini
+++ b/tox.ini
@@ -35,7 +35,7 @@ commands =
     pytest -rsx --cov --cov-config setup.cfg \
       --cov-report=html:{envdir}/htmlcov --cov-context=test \
       --cov-report=xml:{toxworkdir}/coverage.{envname}.xml \
-      tests {posargs:--run-integration}
+      tests {posargs}
 
 [testenv:fix]
 description = run static analysis and style checks
@@ -52,8 +52,7 @@ commands =
 
 [testenv:type]
 description = run type check on code base
-deps =
-    mypy==0.782
+extras = typing
 commands =
     mypy --python-version 3.8 src/build
     mypy --python-version 2.7 src/build


Test patch:
diff --git a/tests/test_env.py b/tests/test_env.py
index 969eff4..9a588ac 100644
--- a/tests/test_env.py
+++ b/tests/test_env.py
@@ -1,8 +1,8 @@
 # SPDX-License-Identifier: MIT
 import json
 import os
-import os.path
 import platform
+import shutil
 import subprocess
 import sys
 
@@ -14,23 +14,15 @@ import build.env
 @pytest.mark.isolated
 def test_isolation():
     subprocess.check_call([sys.executable, '-c', 'import build.env'])
-    with build.env.IsolatedEnvironment.for_current() as env:
+    with build.env.IsolatedEnvBuilder() as env:
         with pytest.raises(subprocess.CalledProcessError):
             debug = 'import sys; import os; print(os.linesep.join(sys.path));'
             subprocess.check_call([env.executable, '-c', '{} import build.env'.format(debug)])
 
 
-@pytest.mark.isolated
-def test_isolated_environment_setup_require_virtualenv(mocker):
-    mocker.patch.dict(os.environ, {'PIP_REQUIRE_VIRTUALENV': 'true'})
-    with build.env.IsolatedEnvironment.for_current():
-        assert 'PIP_REQUIRE_VIRTUALENV' not in os.environ
-    assert os.environ['PIP_REQUIRE_VIRTUALENV'] == 'true'
-
-
 @pytest.mark.isolated
 def test_isolated_environment_install(mocker):
-    with build.env.IsolatedEnvironment.for_current() as env:
+    with build.env.IsolatedEnvBuilder() as env:
         mocker.patch('subprocess.check_call')
 
         env.install([])
@@ -42,7 +34,7 @@ def test_isolated_environment_install(mocker):
         args = subprocess.check_call.call_args[0][0][:-1]
         assert args == [
             env._pip_executable,
-            '-{}m'.format('E' if env._pip_executable == env._executable and sys.version_info[0] == 2 else ''),
+            '-{}m'.format('E' if env._pip_executable == env._python_executable and sys.version_info[0] == 2 else ''),
             'pip',
             'install',
             '--prefix',
@@ -53,19 +45,12 @@ def test_isolated_environment_install(mocker):
         ]
 
 
-def test_uninitialised_isolated_environment():
-    env = build.env.IsolatedEnvironment.for_current()
-
-    with pytest.raises(RuntimeError):
-        env.path
-
-
 @pytest.mark.isolated
 def test_create_isolated_build_host_with_no_pip(tmp_path, capfd, mocker):
     mocker.patch.object(build.env, 'pip', None)
     expected = {'pip', 'greenlet', 'readline', 'cffi'} if platform.python_implementation() == 'PyPy' else {'pip'}
 
-    with build.env.IsolatedEnvironment.for_current() as isolated_env:
+    with build.env.IsolatedEnvBuilder() as isolated_env:
         cmd = [isolated_env.executable, '-m', 'pip', 'list', '--format', 'json']
         packages = {p['name'] for p in json.loads(subprocess.check_output(cmd, universal_newlines=True))}
         assert packages == expected
@@ -80,9 +65,61 @@ def test_create_isolated_build_host_with_no_pip(tmp_path, capfd, mocker):
 
 @pytest.mark.isolated
 def test_create_isolated_build_has_with_pip(tmp_path, capfd, mocker):
-    with build.env.IsolatedEnvironment.for_current() as isolated_env:
+    with build.env.IsolatedEnvBuilder() as isolated_env:
         pass
     assert isolated_env._pip_executable == sys.executable
     out, err = capfd.readouterr()
     assert not out
     assert not err
+
+
+@pytest.mark.skipif(sys.version_info[0] == 2, reason='venv module used on Python 3 only')
+def test_fail_to_get_script_path(mocker):
+    get_path = mocker.patch('sysconfig.get_path', return_value=None)
+    with pytest.raises(RuntimeError, match="Couldn't get environment scripts path"):
+        env = build.env.IsolatedEnvBuilder()
+        with env:
+            pass
+    assert not os.path.exists(env._path)
+    assert get_path.call_count == 1
+
+
+@pytest.mark.skipif(sys.version_info[0] == 2, reason='venv module used on Python 3 only')
+def test_executable_missing_post_creation(mocker):
+    import sysconfig
+
+    original_get_path = sysconfig.get_path
+
+    def _get_path(name, vars):  # noqa
+        shutil.rmtree(vars['base'])
+        return original_get_path(name, vars=vars)
+
+    get_path = mocker.patch('sysconfig.get_path', side_effect=_get_path)
+    with pytest.raises(RuntimeError, match='Virtual environment creation failed, executable .* missing'):
+        with build.env.IsolatedEnvBuilder():
+            pass
+    assert get_path.call_count == 1
+
+
+def test_isolated_env_abstract():
+    with pytest.raises(TypeError):
+        build.env.IsolatedEnv()
+
+
+def test_isolated_env_has_executable_still_abstract():
+    class Env(build.env.IsolatedEnv):  # noqa
+        @property
+        def executable(self):
+            raise NotImplementedError
+
+    with pytest.raises(TypeError):
+        Env()
+
+
+def test_isolated_env_has_install_still_abstract():
+    class Env(build.env.IsolatedEnv):  # noqa
+        def install(self, requirements):
+            raise NotImplementedError
+
+    with pytest.raises(TypeError):
+        Env()
diff --git a/tests/test_integration.py b/tests/test_integration.py
index 573ae62..cd48b0c 100644
--- a/tests/test_integration.py
+++ b/tests/test_integration.py
@@ -113,8 +113,8 @@ def test_build(monkeypatch, project, args, call, tmp_path):
         subprocess.check_call(call + args)
 
     pkg_names = os.listdir(str(pkgs))
-    assert filter(_SDIST.match, pkg_names)
-    assert filter(_WHEEL.match, pkg_names)
+    assert list(filter(_SDIST.match, pkg_names))
+    assert list(filter(_WHEEL.match, pkg_names))
 
 
 def test_isolation(tmp_dir, test_flit_path, mocker):
diff --git a/tests/test_main.py b/tests/test_main.py
index bc93a51..bca03a1 100644
--- a/tests/test_main.py
+++ b/tests/test_main.py
@@ -87,39 +87,56 @@ def test_prog():
 
 
 @pytest.mark.isolated
-def test_build(mocker, test_flit_path):
-    mocker.patch('importlib.import_module', autospec=True)
-    mocker.patch('build.ProjectBuilder.check_dependencies')
-    mocker.patch('build.ProjectBuilder.build')
+def test_build_isolated(mocker, test_flit_path):
+    build_cmd = mocker.patch('build.ProjectBuilder.build')
     mocker.patch('build.__main__._error')
-    mocker.patch('build.env.IsolatedEnvironment.install')
+    install = mocker.patch('build.env._IsolatedEnvVenvPip.install')
 
-    build.ProjectBuilder.check_dependencies.side_effect = [[], ['something'], [], []]
-    build.env.IsolatedEnvironment._path = mocker.Mock()
-
-    # isolation=True
     build.__main__.build(test_flit_path, '.', ['sdist'])
-    build.ProjectBuilder.build.assert_called_with('sdist', '.')
 
+    build_cmd.assert_called_with('sdist', '.')
+    install.assert_called_with({'flit_core >=2,<3'})
+
+
+def test_build_no_isolation_check_deps_empty(mocker, test_flit_path):
     # check_dependencies = []
+    build_cmd = mocker.patch('build.ProjectBuilder.build')
+    mocker.patch('build.ProjectBuilder.check_dependencies', return_value=[])
+
     build.__main__.build(test_flit_path, '.', ['sdist'], isolation=False)
-    build.ProjectBuilder.build.assert_called_with('sdist', '.')
-    build.env.IsolatedEnvironment.install.assert_called_with({'flit_core >=2,<3'})
 
+    build_cmd.assert_called_with('sdist', '.')
+
+
+def test_build_no_isolation_with_check_deps(mocker, test_flit_path):
     # check_dependencies = ['something']
+    error = mocker.patch('build.__main__._error')
+    build_cmd = mocker.patch('build.ProjectBuilder.build')
+    mocker.patch('build.ProjectBuilder.check_dependencies', return_value=['something'])
+
     build.__main__.build(test_flit_path, '.', ['sdist'], isolation=False)
-    build.ProjectBuilder.build.assert_called_with('sdist', '.')
-    build.__main__._error.assert_called_with('Missing dependencies:\n\tsomething')
 
-    build.ProjectBuilder.build.side_effect = [build.BuildException, build.BuildBackendException]
-    build.__main__._error.reset_mock()
+    build_cmd.assert_called_with('sdist', '.')
+    error.assert_called_with('Missing dependencies:\n\tsomething')
+
+
+@pytest.mark.isolated
+def test_build_raises_build_exception(mocker, test_flit_path):
+    error = mocker.patch('build.__main__._error')
+    mocker.patch('build.ProjectBuilder.build', side_effect=build.BuildException)
+    mocker.patch('build.env._IsolatedEnvVenvPip.install')
 
-    # BuildException
     build.__main__.build(test_flit_path, '.', ['sdist'])
-    build.__main__._error.assert_called_with('')
 
-    build.__main__._error.reset_mock()
+    error.assert_called_with('')
+
+
+@pytest.mark.isolated
+def test_build_raises_build_backend_exception(mocker, test_flit_path):
+    error = mocker.patch('build.__main__._error')
+    mocker.patch('build.ProjectBuilder.build', side_effect=build.BuildBackendException)
+    mocker.patch('build.env._IsolatedEnvVenvPip.install')
 
-    # BuildBackendException
     build.__main__.build(test_flit_path, '.', ['sdist'])
-    build.__main__._error.assert_called_with('')
+
+    error.assert_called_with('')
diff --git a/tests/test_projectbuilder.py b/tests/test_projectbuilder.py
index 640c25d..0c0a036 100644
--- a/tests/test_projectbuilder.py
+++ b/tests/test_projectbuilder.py
@@ -88,16 +88,18 @@ def test_init(mocker, test_flit_path, legacy_path, test_no_permission, test_bad_
         'setuptools.build_meta:__legacy__': None,
     }
     mocker.patch('importlib.import_module', modules.get)
-    mocker.patch('pep517.wrappers.Pep517HookCaller')
+    caller = mocker.patch('pep517.wrappers.Pep517HookCaller')
 
     # correct flit pyproject.toml
     build.ProjectBuilder(test_flit_path)
-    pep517.wrappers.Pep517HookCaller.assert_called_with(test_flit_path, 'flit_core.buildapi', backend_path=None)
-    pep517.wrappers.Pep517HookCaller.reset_mock()
+    caller.assert_called_with(test_flit_path, 'flit_core.buildapi', backend_path=None, python_executable=sys.executable)
+    caller.reset_mock()
 
     # FileNotFoundError
     build.ProjectBuilder(legacy_path)
-    pep517.wrappers.Pep517HookCaller.assert_called_with(legacy_path, 'setuptools.build_meta:__legacy__', backend_path=None)
+    caller.assert_called_with(
+        legacy_path, 'setuptools.build_meta:__legacy__', backend_path=None, python_executable=sys.executable
+    )
 
     # PermissionError
     if sys.version_info[0] != 2 and os.name != 'nt':  # can't correctly set the permissions required for this