Repository: tox-dev/tox (python)

Issue:
Is it Tox or tox?
In a simpler time I did not care, but know I realize I am not sure and I like consistency, which at the moment is not given. At the moment Tox and tox is used both e.g. in the [docs](http://tox.readthedocs.io/en/latest/). I would say it should be tox, just like e.g. pytest and devpi. 

Agreed?

I would update the docs then and try to use it consistently in all communication.

P.S. I know, I can be a bit of a pedant sometimes :)

Pull request:
Tox -> tox
fixes #589

Patch:
diff --git a/CHANGELOG b/CHANGELOG
index 96867575..e0c65960 100644
--- a/CHANGELOG
+++ b/CHANGELOG
@@ -1,4 +1,11 @@
-2.8rc2 (2017-08-26)
+2.8.1 (2017-09-04)
+------------------
+
+- #p599: fix problems with implementation of #515.
+         Substitutions from other sections were not made anymore if they were not in `envlist`.
+         Thanks to Clark Boylan (@cboylan) for helping to get this fixed (#p597).
+
+2.8.0 (2017-09-01)
 -------------------
 
 - #276: Remove easy_install from docs (TL;DR: use pip).
diff --git a/README.rst b/README.rst
index 8d0b92e6..db8d6e39 100644
--- a/README.rst
+++ b/README.rst
@@ -1,7 +1,5 @@
-
-What is Tox?
---------------------
-
+tox automation project
+----------------------
 
 .. image:: https://img.shields.io/pypi/v/tox.svg
    :target: https://pypi.org/project/tox/
@@ -12,8 +10,14 @@ What is Tox?
 .. image:: https://img.shields.io/appveyor/ci/RonnyPfannschmidt/tox/master.svg
     :target: https://ci.appveyor.com/project/RonnyPfannschmidt/tox
 
+**vision: standardize testing in Python**
+
+tox aims to automate and standardize testing in Python. It is part of a larger vision of easing the packaging, testing and release process of Python software.
+
+What is tox?
+------------
 
-Tox is a generic virtualenv management and test command line tool you can use for:
+tox is a generic virtualenv management and test command line tool you can use for:
 
 * checking your package installs correctly with different Python versions and
   interpreters
@@ -23,7 +27,7 @@ Tox is a generic virtualenv management and test command line tool you can use fo
 * acting as a frontend to Continuous Integration servers, greatly
   reducing boilerplate and merging CI and shell-based testing.
 
-For more information and the repository please checkout:
+For more information and the repository please see:
 
 - home and docs: https://tox.readthedocs.org
 
diff --git a/doc/config-v2.txt b/doc/config-v2.txt
index 288586ab..cb6e5e87 100644
--- a/doc/config-v2.txt
+++ b/doc/config-v2.txt
@@ -18,7 +18,7 @@ their python projects.
 Issues with current tox (1.4) configuration
 ------------------------------------------------
 
-Tox is used as a tool for creating and managing virtualenv environments
+tox is used as a tool for creating and managing virtualenv environments
 and running tests in them. As of tox-1.4 there are some issues frequently
 coming up with its configuration language:
 
@@ -124,7 +124,7 @@ Nothing special here :)
 
 .. note::
 
-    Tox provides good defaults for platform and basepython
+    tox provides good defaults for platform and basepython
     settings, so the above ini-file can be further reduced::
 
         [tox]
diff --git a/doc/config.txt b/doc/config.txt
index 8880123c..32699aad 100644
--- a/doc/config.txt
+++ b/doc/config.txt
@@ -9,7 +9,7 @@ tox configuration specification
 Below you find the specification, but you might want to skim some
 :doc:`examples` first and use this page as a reference.
 
-Tox global settings
+tox global settings
 -------------------
 
 List of optional global options::
@@ -135,8 +135,8 @@ Complete list of settings that you can put into ``testenv*`` sections:
       similarity in function.
 
       Note that in tox 2.0, the default behavior of tox with respect to
-      treating errors from commands changed. Tox < 2.0 would ignore errors by
-      default. Tox >= 2.0 will abort on an error by default, which is safer and more
+      treating errors from commands changed. tox < 2.0 would ignore errors by
+      default. tox >= 2.0 will abort on an error by default, which is safer and more
       typical of CI and command execution tools, as it doesn't make sense to
       run tests if installing some prerequisite failed and it doesn't make sense to
       try to deploy if tests failed.
@@ -615,7 +615,7 @@ Reading it line by line:
 
 .. note::
 
-    Tox provides good defaults for basepython setting, so the above
+    tox provides good defaults for basepython setting, so the above
     ini-file can be further reduced by omitting the ``basepython``
     setting.
 
diff --git a/doc/example/basic.txt b/doc/example/basic.txt
index d867fb65..0775b720 100644
--- a/doc/example/basic.txt
+++ b/doc/example/basic.txt
@@ -227,12 +227,12 @@ special handling of PYTHONHASHSEED
 
 .. versionadded:: 1.6.2
 
-By default, Tox sets PYTHONHASHSEED_ for test commands to a random integer
+By default, tox sets PYTHONHASHSEED_ for test commands to a random integer
 generated when ``tox`` is invoked.  This mimics Python's hash randomization
 enabled by default starting `in Python 3.3`_.  To aid in reproducing test
-failures, Tox displays the value of ``PYTHONHASHSEED`` in the test output.
+failures, tox displays the value of ``PYTHONHASHSEED`` in the test output.
 
-You can tell Tox to use an explicit hash seed value via the ``--hashseed``
+You can tell tox to use an explicit hash seed value via the ``--hashseed``
 command-line option to ``tox``.  You can also override the hash seed value
 per test environment in ``tox.ini`` as follows::
 
diff --git a/doc/example/devenv.txt b/doc/example/devenv.txt
index f4afc375..07475d0b 100644
--- a/doc/example/devenv.txt
+++ b/doc/example/devenv.txt
@@ -2,7 +2,7 @@
 Development environment
 =======================
 
-Tox can be used for just preparing different virtual environments required by a
+tox can be used for just preparing different virtual environments required by a
 project.
 
 This feature can be used by deployment tools when preparing deployed project
diff --git a/doc/example/general.txt b/doc/example/general.txt
index eb14a44e..c0167449 100644
--- a/doc/example/general.txt
+++ b/doc/example/general.txt
@@ -113,7 +113,7 @@ the following entry::
     deps=
         {distshare}/one-*.zip  # install latest package from "one" project
 
-That's all.  Tox running on project ``one`` will copy the sdist-package
+That's all.  tox running on project ``one`` will copy the sdist-package
 into the ``distshare`` directory after which a ``tox`` run on project
 ``two`` will grab it because ``deps`` contain an entry with the
 ``one-*.zip`` pattern.  If there is more than one matching package the
diff --git a/doc/example/jenkins.txt b/doc/example/jenkins.txt
index 260d14e8..0cd24ae6 100644
--- a/doc/example/jenkins.txt
+++ b/doc/example/jenkins.txt
@@ -1,5 +1,5 @@
 
-Using Tox with the Jenkins Integration Server
+Using tox with the Jenkins Integration Server
 =================================================
 
 Using Jenkins multi-configuration jobs
diff --git a/doc/index.txt b/doc/index.txt
index 1099bd4d..3777925b 100644
--- a/doc/index.txt
+++ b/doc/index.txt
@@ -8,10 +8,10 @@ vision: standardize testing in Python
 of a larger vision of easing the packaging, testing and release process
 of Python software.
 
-What is Tox?
+What is tox?
 --------------------
 
-Tox is a generic virtualenv_ management and test command line tool you can use for:
+tox is a generic virtualenv_ management and test command line tool you can use for:
 
 * checking your package installs correctly with different Python versions and
   interpreters
diff --git a/tox/__init__.py b/tox/__init__.py
index 73d095eb..98f1246a 100644
--- a/tox/__init__.py
+++ b/tox/__init__.py
@@ -14,12 +14,18 @@ class exception:
         def __str__(self):
             return "%s: %s" % (self.__class__.__name__, self.args[0])
 
+    class MissingSubstitution(Exception):
+        FLAG = 'TOX_MISSING_SUBSTITUTION'
+        """placeholder for debugging configurations"""
+        def __init__(self, name):
+            self.name = name
+
     class ConfigError(Error):
         """ error in tox configuration. """
     class UnsupportedInterpreter(Error):
-        "signals an unsupported Interpreter"
+        """signals an unsupported Interpreter"""
     class InterpreterNotFound(Error):
-        "signals that an interpreter could not be found"
+        """signals that an interpreter could not be found"""
     class InvocationError(Error):
         """ an error while invoking a script. """
     class MissingFile(Error):
@@ -35,4 +41,5 @@ class exception:
             self.message = message
             super(exception.MinVersionError, self).__init__(message)
 
+
 from tox.session import main as cmdline  # noqa
diff --git a/tox/_quickstart.py b/tox/_quickstart.py
index bc283a8a..e61c1a95 100644
--- a/tox/_quickstart.py
+++ b/tox/_quickstart.py
@@ -62,7 +62,7 @@ all_envs = ['py26', 'py27', 'py32', 'py33', 'py34', 'py35', 'py36', 'pypy', 'jyt
 PROMPT_PREFIX = '> '
 
 QUICKSTART_CONF = '''\
-# Tox (https://tox.readthedocs.io/) is a tool for running tests
+# tox (https://tox.readthedocs.io/) is a tool for running tests
 # in multiple virtualenvs. This configuration file will run the
 # test suite on all supported python versions. To use it, "pip install tox"
 # and then run "tox" from this directory.
@@ -148,7 +148,7 @@ def ask_user(d):
 
     """
 
-    print('Welcome to the Tox %s quickstart utility.' % __version__)
+    print('Welcome to the tox %s quickstart utility.' % __version__)
     print('''
 This utility will ask you a few questions and then generate a simple tox.ini
 file to help get you started using tox.
diff --git a/tox/config.py b/tox/config.py
index a15b69bb..d5a6baf8 100755
--- a/tox/config.py
+++ b/tox/config.py
@@ -609,6 +609,13 @@ class TestenvConfig:
         #: set of factors
         self.factors = factors
         self._reader = reader
+        self.missing_subs = []
+        """Holds substitutions that could not be resolved.
+
+        Pre 2.8.1 missing substitutions crashed with a ConfigError although this would not be a
+        problem if the env is not part of the current testrun. So we need to remember this and
+        check later when the testenv is actually run and crash only then.
+        """
 
     def get_envbindir(self):
         """ path to directory where scripts/binaries reside. """
@@ -791,9 +798,7 @@ class parseini:
             section = testenvprefix + name
             factors = set(name.split('-'))
             if section in self._cfg or factors <= known_factors:
-                config.envconfigs[name] = \
-                    self.make_envconfig(name, section, reader._subs, config,
-                                        replace=name in config.envlist)
+                config.envconfigs[name] = self.make_envconfig(name, section, reader._subs, config)
 
         all_develop = all(name in config.envconfigs
                           and config.envconfigs[name].usedevelop
@@ -813,33 +818,31 @@ class parseini:
         factors = set(name.split('-'))
         reader = SectionReader(section, self._cfg, fallbacksections=["testenv"],
                                factors=factors)
-        vc = TestenvConfig(config=config, envname=name, factors=factors, reader=reader)
-        reader.addsubstitutions(**subs)
-        reader.addsubstitutions(envname=name)
-        reader.addsubstitutions(envbindir=vc.get_envbindir,
-                                envsitepackagesdir=vc.get_envsitepackagesdir,
-                                envpython=vc.get_envpython)
-
+        tc = TestenvConfig(name, config, factors, reader)
+        reader.addsubstitutions(
+            envname=name, envbindir=tc.get_envbindir, envsitepackagesdir=tc.get_envsitepackagesdir,
+            envpython=tc.get_envpython, **subs)
         for env_attr in config._testenv_attr:
             atype = env_attr.type
-            if atype in ("bool", "path", "string", "dict", "dict_setenv", "argv", "argvlist"):
-                meth = getattr(reader, "get" + atype)
-                res = meth(env_attr.name, env_attr.default, replace=replace)
-            elif atype == "space-separated-list":
-                res = reader.getlist(env_attr.name, sep=" ")
-            elif atype == "line-list":
-                res = reader.getlist(env_attr.name, sep="\n")
-            else:
-                raise ValueError("unknown type %r" % (atype,))
-
-            if env_attr.postprocess:
-                res = env_attr.postprocess(testenv_config=vc, value=res)
-            setattr(vc, env_attr.name, res)
-
+            try:
+                if atype in ("bool", "path", "string", "dict", "dict_setenv", "argv", "argvlist"):
+                    meth = getattr(reader, "get" + atype)
+                    res = meth(env_attr.name, env_attr.default, replace=replace)
+                elif atype == "space-separated-list":
+                    res = reader.getlist(env_attr.name, sep=" ")
+                elif atype == "line-list":
+                    res = reader.getlist(env_attr.name, sep="\n")
+                else:
+                    raise ValueError("unknown type %r" % (atype,))
+                if env_attr.postprocess:
+                    res = env_attr.postprocess(testenv_config=tc, value=res)
+            except tox.exception.MissingSubstitution as e:
+                tc.missing_subs.append(e.name)
+                res = e.FLAG
+            setattr(tc, env_attr.name, res)
             if atype in ("path", "string"):
                 reader.addsubstitutions(**{env_attr.name: res})
-
-        return vc
+        return tc
 
     def _getenvdata(self, reader):
         envstr = self.config.option.env                                \
@@ -957,16 +960,16 @@ class SectionReader:
 
     def getdict(self, name, default=None, sep="\n", replace=True):
         value = self.getstring(name, None, replace=replace)
-        return self._getdict(value, default=default, sep=sep)
+        return self._getdict(value, default=default, sep=sep, replace=replace)
 
     def getdict_setenv(self, name, default=None, sep="\n", replace=True):
         value = self.getstring(name, None, replace=replace, crossonly=True)
-        definitions = self._getdict(value, default=default, sep=sep)
+        definitions = self._getdict(value, default=default, sep=sep, replace=replace)
         self._setenv = SetenvDict(definitions, reader=self)
         return self._setenv
 
-    def _getdict(self, value, default, sep):
-        if value is None:
+    def _getdict(self, value, default, sep, replace=True):
+        if value is None or not replace:
             return default or {}
 
         d = {}
@@ -979,7 +982,7 @@ class SectionReader:
 
     def getbool(self, name, default=None, replace=True):
         s = self.getstring(name, default, replace=replace)
-        if not s:
+        if not s or not replace:
             s = default
         if s is None:
             raise KeyError("no config value [%s] %s found" % (
@@ -1041,9 +1044,15 @@ class SectionReader:
         section_name = section_name if section_name else self.section_name
         self._subststack.append((section_name, name))
         try:
-            return Replacer(self, crossonly=crossonly).do_replace(value)
-        finally:
+            replaced = Replacer(self, crossonly=crossonly).do_replace(value)
             assert self._subststack.pop() == (section_name, name)
+        except tox.exception.MissingSubstitution:
+            if not section_name.startswith(testenvprefix):
+                raise tox.exception.ConfigError(
+                    "substitution env:%r: unknown or recursive definition in "
+                    "section %r." % (value, section_name))
+            raise
+        return replaced
 
 
 class Replacer:
@@ -1111,20 +1120,14 @@ class Replacer:
     def _replace_env(self, match):
         envkey = match.group('substitution_value')
         if not envkey:
-            raise tox.exception.ConfigError(
-                'env: requires an environment variable name')
-
+            raise tox.exception.ConfigError('env: requires an environment variable name')
         default = match.group('default_value')
-
         envvalue = self.reader.get_environ_value(envkey)
-        if envvalue is None:
-            if default is None:
-                raise tox.exception.ConfigError(
-                    "substitution env:%r: unknown environment variable %r "
-                    " or recursive definition." %
-                    (envkey, envkey))
+        if envvalue is not None:
+            return envvalue
+        if default is not None:
             return default
-        return envvalue
+        raise tox.exception.MissingSubstitution(envkey)
 
     def _substitute_from_other_section(self, key):
         if key.startswith("[") and "]" in key:
@@ -1157,7 +1160,7 @@ class _ArgvlistReader:
     def getargvlist(cls, reader, value, replace=True):
         """Parse ``commands`` argvlist multiline string.
 
-        :param str name: Key name in a section.
+        :param SectionReader reader: reader to be used.
         :param str value: Content stored by key.
 
         :rtype: list[list[str]]
diff --git a/tox/session.py b/tox/session.py
index 269a7705..60baaeee 100644
--- a/tox/session.py
+++ b/tox/session.py
@@ -440,6 +440,12 @@ class Session:
             path.ensure(dir=1)
 
     def setupenv(self, venv):
+        if venv.envconfig.missing_subs:
+            venv.status = (
+                "unresolvable substitution(s): %s. "
+                "Environment variables are missing or defined recursively." %
+                (','.join(["'%s'" % m for m in venv.envconfig.missing_subs])))
+            return
         if not venv.matching_platform():
             venv.status = "platform mismatch"
             return  # we simply omit non-matching platforms


Test patch:
diff --git a/tests/test_config.py b/tests/test_config.py
index d6b44426..2533c962 100644
--- a/tests/test_config.py
+++ b/tests/test_config.py
@@ -344,6 +344,21 @@ class TestIniParserAgainstCommandsKey:
         envconfig = config.envconfigs['python']
         assert envconfig.commands == [["echo", "bar"]]
 
+    def test_regression_issue595(self, newconfig):
+        config = newconfig("""
+            [tox]
+            envlist = foo
+            [testenv]
+            setenv = VAR = x
+            [testenv:bar]
+            setenv = {[testenv]setenv}
+            [testenv:baz]
+            setenv =
+        """)
+        assert config.envconfigs['foo'].setenv['VAR'] == 'x'
+        assert config.envconfigs['bar'].setenv['VAR'] == 'x'
+        assert 'VAR' not in config.envconfigs['baz'].setenv
+
 
 class TestIniParser:
     def test_getstring_single(self, tmpdir, newconfig):
@@ -429,18 +444,20 @@ class TestIniParser:
         x = reader.getdict("key3", {1: 2})
         assert x == {1: 2}
 
-    def test_getstring_environment_substitution(self, monkeypatch, newconfig):
-        monkeypatch.setenv("KEY1", "hello")
-        config = newconfig("""
-            [section]
-            key1={env:KEY1}
-            key2={env:KEY2}
-        """)
-        reader = SectionReader("section", config._cfg)
-        x = reader.getstring("key1")
-        assert x == "hello"
+    def test_normal_env_sub_works(self, monkeypatch, newconfig):
+        monkeypatch.setenv("VAR", "hello")
+        config = newconfig("[section]\nkey={env:VAR}")
+        assert SectionReader("section", config._cfg).getstring("key") == "hello"
+
+    def test_missing_env_sub_raises_config_error_in_non_testenv(self, newconfig):
+        config = newconfig("[section]\nkey={env:VAR}")
         with pytest.raises(tox.exception.ConfigError):
-            reader.getstring("key2")
+            SectionReader("section", config._cfg).getstring("key")
+
+    def test_missing_env_sub_populates_missing_subs(self, newconfig):
+        config = newconfig("[testenv:foo]\ncommands={env:VAR}")
+        print(SectionReader("section", config._cfg).getstring("commands"))
+        assert config.envconfigs['foo'].missing_subs == ['VAR']
 
     def test_getstring_environment_substitution_with_default(self, monkeypatch, newconfig):
         monkeypatch.setenv("KEY1", "hello")
diff --git a/tests/test_quickstart.py b/tests/test_quickstart.py
index 76551008..fd035842 100644
--- a/tests/test_quickstart.py
+++ b/tests/test_quickstart.py
@@ -50,7 +50,7 @@ class TestToxQuickstartMain(object):
         tox._quickstart.main(argv=['tox-quickstart'])
 
         expected_tox_ini = """
-# Tox (https://tox.readthedocs.io/) is a tool for running tests
+# tox (https://tox.readthedocs.io/) is a tool for running tests
 # in multiple virtualenvs. This configuration file will run the
 # test suite on all supported python versions. To use it, "pip install tox"
 # and then run "tox" from this directory.
@@ -92,7 +92,7 @@ deps =
         tox._quickstart.main(argv=['tox-quickstart'])
 
         expected_tox_ini = """
-# Tox (https://tox.readthedocs.io/) is a tool for running tests
+# tox (https://tox.readthedocs.io/) is a tool for running tests
 # in multiple virtualenvs. This configuration file will run the
 # test suite on all supported python versions. To use it, "pip install tox"
 # and then run "tox" from this directory.
@@ -134,7 +134,7 @@ deps =
         tox._quickstart.main(argv=['tox-quickstart'])
 
         expected_tox_ini = """
-# Tox (https://tox.readthedocs.io/) is a tool for running tests
+# tox (https://tox.readthedocs.io/) is a tool for running tests
 # in multiple virtualenvs. This configuration file will run the
 # test suite on all supported python versions. To use it, "pip install tox"
 # and then run "tox" from this directory.
@@ -175,7 +175,7 @@ deps =
         tox._quickstart.main(argv=['tox-quickstart'])
 
         expected_tox_ini = """
-# Tox (https://tox.readthedocs.io/) is a tool for running tests
+# tox (https://tox.readthedocs.io/) is a tool for running tests
 # in multiple virtualenvs. This configuration file will run the
 # test suite on all supported python versions. To use it, "pip install tox"
 # and then run "tox" from this directory.
@@ -208,7 +208,7 @@ deps =
         tox._quickstart.main(argv=['tox-quickstart'])
 
         expected_tox_ini = """
-# Tox (https://tox.readthedocs.io/) is a tool for running tests
+# tox (https://tox.readthedocs.io/) is a tool for running tests
 # in multiple virtualenvs. This configuration file will run the
 # test suite on all supported python versions. To use it, "pip install tox"
 # and then run "tox" from this directory.
@@ -241,7 +241,7 @@ deps =
         tox._quickstart.main(argv=['tox-quickstart'])
 
         expected_tox_ini = """
-# Tox (https://tox.readthedocs.io/) is a tool for running tests
+# tox (https://tox.readthedocs.io/) is a tool for running tests
 # in multiple virtualenvs. This configuration file will run the
 # test suite on all supported python versions. To use it, "pip install tox"
 # and then run "tox" from this directory.
@@ -274,7 +274,7 @@ deps =
         tox._quickstart.main(argv=['tox-quickstart'])
 
         expected_tox_ini = """
-# Tox (https://tox.readthedocs.io/) is a tool for running tests
+# tox (https://tox.readthedocs.io/) is a tool for running tests
 # in multiple virtualenvs. This configuration file will run the
 # test suite on all supported python versions. To use it, "pip install tox"
 # and then run "tox" from this directory.
@@ -316,7 +316,7 @@ deps =
         tox._quickstart.main(argv=['tox-quickstart'])
 
         expected_tox_ini = """
-# Tox (https://tox.readthedocs.io/) is a tool for running tests
+# tox (https://tox.readthedocs.io/) is a tool for running tests
 # in multiple virtualenvs. This configuration file will run the
 # test suite on all supported python versions. To use it, "pip install tox"
 # and then run "tox" from this directory.
@@ -363,7 +363,7 @@ deps =
         tox._quickstart.main(argv=['tox-quickstart'])
 
         expected_tox_ini = """
-# Tox (https://tox.readthedocs.io/) is a tool for running tests
+# tox (https://tox.readthedocs.io/) is a tool for running tests
 # in multiple virtualenvs. This configuration file will run the
 # test suite on all supported python versions. To use it, "pip install tox"
 # and then run "tox" from this directory.
@@ -402,7 +402,7 @@ deps =
         assert tox_ini_path.isfile()
 
         expected_tox_ini = """
-# Tox (https://tox.readthedocs.io/) is a tool for running tests
+# tox (https://tox.readthedocs.io/) is a tool for running tests
 # in multiple virtualenvs. This configuration file will run the
 # test suite on all supported python versions. To use it, "pip install tox"
 # and then run "tox" from this directory.
@@ -444,7 +444,7 @@ deps =
         assert tox_ini_path.isfile()
 
         expected_tox_ini = """
-# Tox (https://tox.readthedocs.io/) is a tool for running tests
+# tox (https://tox.readthedocs.io/) is a tool for running tests
 # in multiple virtualenvs. This configuration file will run the
 # test suite on all supported python versions. To use it, "pip install tox"
 # and then run "tox" from this directory.
@@ -493,7 +493,7 @@ class TestToxQuickstart(object):
             'deps': 'pytest',
         }
         expected_tox_ini = """
-# Tox (https://tox.readthedocs.io/) is a tool for running tests
+# tox (https://tox.readthedocs.io/) is a tool for running tests
 # in multiple virtualenvs. This configuration file will run the
 # test suite on all supported python versions. To use it, "pip install tox"
 # and then run "tox" from this directory.
@@ -520,7 +520,7 @@ deps =
             'deps': '',
         }
         expected_tox_ini = """
-# Tox (https://tox.readthedocs.io/) is a tool for running tests
+# tox (https://tox.readthedocs.io/) is a tool for running tests
 # in multiple virtualenvs. This configuration file will run the
 # test suite on all supported python versions. To use it, "pip install tox"
 # and then run "tox" from this directory.
@@ -546,7 +546,7 @@ deps =
             'deps': 'Twisted',
         }
         expected_tox_ini = """
-# Tox (https://tox.readthedocs.io/) is a tool for running tests
+# tox (https://tox.readthedocs.io/) is a tool for running tests
 # in multiple virtualenvs. This configuration file will run the
 # test suite on all supported python versions. To use it, "pip install tox"
 # and then run "tox" from this directory.
@@ -577,7 +577,7 @@ deps =
             'deps': 'nose',
         }
         expected_tox_ini = """
-# Tox (https://tox.readthedocs.io/) is a tool for running tests
+# tox (https://tox.readthedocs.io/) is a tool for running tests
 # in multiple virtualenvs. This configuration file will run the
 # test suite on all supported python versions. To use it, "pip install tox"
 # and then run "tox" from this directory.
diff --git a/tests/test_venv.py b/tests/test_venv.py
index 3cd544f4..ad0058b0 100644
--- a/tests/test_venv.py
+++ b/tests/test_venv.py
@@ -593,6 +593,8 @@ class TestVenvTest:
         assert l[1].env['PYTHONPATH'] == '/my/awesome/library'
 
 
+# FIXME this test fails when run in isolation - find what this depends on
+# AssertionError: found warning('*Discarding $PYTHONPATH [...]
 def test_env_variables_added_to_pcall(tmpdir, mocksession, newconfig, monkeypatch):
     pkg = tmpdir.ensure("package.tar.gz")
     monkeypatch.setenv("X123", "123")
diff --git a/tests/test_z_cmdline.py b/tests/test_z_cmdline.py
index 49e7641f..97e0712b 100644
--- a/tests/test_z_cmdline.py
+++ b/tests/test_z_cmdline.py
@@ -871,3 +871,10 @@ def test_envtmpdir(initproj, cmd):
 
     result = cmd.run("tox")
     assert not result.ret
+
+
+def test_missing_env_fails(initproj, cmd):
+    initproj("foo", filedefs={'tox.ini': "[testenv:foo]\ncommands={env:VAR}"})
+    result = cmd.run("tox")
+    assert result.ret == 1
+    result.stdout.fnmatch_lines(["*foo: unresolvable substitution(s): 'VAR'*"])