Repository: tox-dev/tox

Issue:
--result-json does not respect --skip-missing-interpreters-option
When executing tox with --result-json and --skip-missing-interpreters-option, the latter seems not to be reflected in the generated json file. Please find the sample project attached below:
[missing_interpreters.tar.gz](https://github.com/tox-dev/tox/files/1474831/missing_interpreters.tar.gz)

To reproduce the issue simply invoke:

`tox --skip-missing-interpreters --result-json toxreport.json`
Tox main process exits with return code 0, but in toxreport.json file user can see the following:
```
"python": {
  "setup": [
  {
    "command": [
    "setup virtualenv"
    ],
    "output": "InterpreterNotFound: xyz_unknown_interpreter",
    "retcode": "1"
   }
   ]
}
```
In my opinion retcode here should be changed to 0 if --skip-missing-interpreters option is explicitly used.

Discussion:
matthiasha: This will make Devpi report those tests as "passed", right?
r2dan: @matthiasha: Yes, setup of those test environments will be "passed" on devpi. 

Patch:
diff --git a/.gitignore b/.gitignore
index f232c593..517aca48 100644
--- a/.gitignore
+++ b/.gitignore
@@ -15,6 +15,7 @@ doc/_build/
 tox.egg-info
 .tox
 .cache
+.python-version
 
 coverage.xml
 htmlcov
diff --git a/CONTRIBUTING.rst b/CONTRIBUTING.rst
index f51af1fa..1217c0b5 100644
--- a/CONTRIBUTING.rst
+++ b/CONTRIBUTING.rst
@@ -101,7 +101,7 @@ Short version
    The test environments above are usually enough to cover most cases locally.
 
 #. Consider the
-   `checklist <https://github.com/Avira/tox/blob/master/.github/PULL_REQUEST_TEMPLATE.md>`_
+   `checklist <https://github.com/tox-dev/tox/blob/master/.github/PULL_REQUEST_TEMPLATE.md>`_
    in the pull request form
 
 Long version
@@ -178,7 +178,7 @@ Here is a simple overview, with tox-specific bits:
     $ git push -u
 
 
-#. submit a pull request through the GitHub website and and consider the `checklist <https://github.com/Avira/tox/blob/master/.github/PULL_REQUEST_TEMPLATE.md>`_ in the pull request form::
+#. submit a pull request through the GitHub website and and consider the `checklist <https://github.com/tox-dev/tox/blob/master/.github/PULL_REQUEST_TEMPLATE.md>`_ in the pull request form::
 
     head-fork: YOUR_GITHUB_USERNAME/tox
     compare: your-branch-name
diff --git a/CONTRIBUTORS b/CONTRIBUTORS
index edb19b1b..36e57397 100644
--- a/CONTRIBUTORS
+++ b/CONTRIBUTORS
@@ -5,6 +5,7 @@ Alexandre Conrad
 Allan Feldman
 Andrii Soldatenko
 Anthon van der Neuth
+Anthony Sottile
 Asmund Grammeltwedt
 Barry Warsaw
 Bartolome Sanchez Salado
@@ -18,12 +19,14 @@ Cyril Roelandt
 Eli Collins
 Eugene Yunak
 Fernando L. Pereira
+Florian Schulze
 Hazal Ozturk
 Henk-Jaap Wagenaar
 Ian Stapleton Cordasco
 Igor Duarte Cardoso
 Ionel Maries Cristian
 Itxaka Serrano
+Jake Windle
 Jannis Leidel
 Johannes Christ
 Josh Smeaton
@@ -36,6 +39,7 @@ Lukasz Rogalski
 Manuel Jacob
 Marc Abramowitz
 Marc Schlaich
+Mariusz Rusiniak
 Mark Hirota
 Matt Good
 Matt Jeffery
diff --git a/changelog/426.bugfix.rst b/changelog/426.bugfix.rst
new file mode 100644
index 00000000..caf55dd2
--- /dev/null
+++ b/changelog/426.bugfix.rst
@@ -0,0 +1,1 @@
+Write directly to stdout buffer if possible to prevent str vs bytes issues - by @asottile
diff --git a/changelog/672.bugfix.rst b/changelog/672.bugfix.rst
new file mode 100644
index 00000000..742a1ce7
--- /dev/null
+++ b/changelog/672.bugfix.rst
@@ -0,0 +1,1 @@
+fix #672 reporting to json file when skip-missing-interpreters option is used - by @r2dan
diff --git a/changelog/723.bugfix.rst b/changelog/723.bugfix.rst
index 8ec40c8e..b6757a19 100644
--- a/changelog/723.bugfix.rst
+++ b/changelog/723.bugfix.rst
@@ -1,5 +1,5 @@
 Fixed an issue where invocation of Tox from the Python package, where
-invocation errors (failed actions) occur results in a change in the 
+invocation errors (failed actions) occur results in a change in the
 sys.stdout stream encoding in Python 3.x.
 New behaviour is that sys.stdout is reset back to its original encoding
 after invocation errors - by @tonybaloney
diff --git a/changelog/727.bugfix.rst b/changelog/727.bugfix.rst
new file mode 100644
index 00000000..f8db4df0
--- /dev/null
+++ b/changelog/727.bugfix.rst
@@ -0,0 +1,1 @@
+The reading of command output sometimes failed with ``IOError: [Errno 0] Error`` on Windows, this was fixed by using a simpler method to update the read buffers. - by @fschulze
diff --git a/doc/example/platform.rst b/doc/example/platform.rst
index 5661fc65..647dc3b3 100644
--- a/doc/example/platform.rst
+++ b/doc/example/platform.rst
@@ -26,7 +26,7 @@ and the following ``tox.ini`` content:
                mymacos: darwin
                mywindows: win32
 
-    # you can specify dependencies and their versions based on platform filtered envirements
+    # you can specify dependencies and their versions based on platform filtered environments
     deps = mylinux,mymacos: py==1.4.32
            mywindows: py==1.4.30
 
diff --git a/setup.py b/setup.py
index 8b5a63f7..4d91d9f6 100644
--- a/setup.py
+++ b/setup.py
@@ -59,7 +59,7 @@ def main():
                                  'towncrier >= 17.8.0'],
                         'lint': ['flake8 == 3.4.1',
                                  'flake8-bugbear == 17.4.0',
-                                 'pre-commit == 1.4.3'],
+                                 'pre-commit == 1.4.4'],
                         'publish': ['devpi',
                                     'twine']},
         classifiers=['Development Status :: 5 - Production/Stable',
diff --git a/tox/session.py b/tox/session.py
index 27547872..f532f212 100644
--- a/tox/session.py
+++ b/tox/session.py
@@ -135,7 +135,7 @@ class Action(object):
             fout.write("actionid: %s\nmsg: %s\ncmdargs: %r\n\n" % (self.id, self.msg, args))
             fout.flush()
             outpath = py.path.local(fout.name)
-            fin = outpath.open()
+            fin = outpath.open('rb')
             fin.read()  # read the header, so it won't be written to stdout
             stdout = fout
         elif returnout:
@@ -159,21 +159,25 @@ class Action(object):
             self.report.logpopen(popen, env=env)
             try:
                 if resultjson and not redirect:
-                    assert popen.stderr is None  # prevent deadlock
+                    if popen.stderr is not None:
+                        # prevent deadlock
+                        raise ValueError("stderr must not be piped here")
+                    # we read binary from the process and must write using a
+                    # binary stream
+                    buf = getattr(sys.stdout, 'buffer', sys.stdout)
                     out = None
                     last_time = time.time()
                     while 1:
-                        fin_pos = fin.tell()
                         # we have to read one byte at a time, otherwise there
                         # might be no output for a long time with slow tests
                         data = fin.read(1)
                         if data:
-                            sys.stdout.write(data)
-                            if '\n' in data or (time.time() - last_time) > 1:
+                            buf.write(data)
+                            if b'\n' in data or (time.time() - last_time) > 1:
                                 # we flush on newlines or after 1 second to
                                 # provide quick enough feedback to the user
                                 # when printing a dot per test
-                                sys.stdout.flush()
+                                buf.flush()
                                 last_time = time.time()
                         elif popen.poll() is not None:
                             if popen.stdout is not None:
@@ -181,7 +185,8 @@ class Action(object):
                             break
                         else:
                             time.sleep(0.1)
-                            fin.seek(fin_pos)
+                            # the seek updates internal read buffers
+                            fin.seek(0, 1)
                     fin.close()
                 else:
                     out, err = popen.communicate()
@@ -478,6 +483,7 @@ class Session:
         action = self.newaction(venv, "getenv", venv.envconfig.envdir)
         with action:
             venv.status = 0
+            default_ret_code = 1
             envlog = self.resultlog.get_envlog(venv.name)
             try:
                 status = venv.update(action=action)
@@ -492,11 +498,18 @@ class Session:
                         "Error creating virtualenv. Note that some special "
                         "characters (e.g. ':' and unicode symbols) in paths are "
                         "not supported by virtualenv. Error details: %r" % e)
+            except tox.exception.InterpreterNotFound as e:
+                status = e
+                if self.config.option.skip_missing_interpreters:
+                    default_ret_code = 0
             if status:
                 commandlog = envlog.get_commandlog("setup")
-                commandlog.add_command(["setup virtualenv"], str(status), 1)
+                commandlog.add_command(["setup virtualenv"], str(status), default_ret_code)
                 venv.status = status
-                self.report.error(str(status))
+                if default_ret_code == 0:
+                    self.report.skip(str(status))
+                else:
+                    self.report.error(str(status))
                 return False
             commandpath = venv.getcommandpath("python")
             envlog.set_python_info(commandpath)
diff --git a/tox/venv.py b/tox/venv.py
index 3b3ecb32..69571637 100755
--- a/tox/venv.py
+++ b/tox/venv.py
@@ -149,9 +149,6 @@ class VirtualEnv(object):
                     return True
         return False
 
-    def _ispython3(self):
-        return "python3" in str(self.envconfig.basepython)
-
     def update(self, action):
         """ return status string for updating actual venv to match configuration.
             if status string is empty, all is ok.
@@ -170,8 +167,6 @@ class VirtualEnv(object):
             self.just_created = True
         except tox.exception.UnsupportedInterpreter:
             return sys.exc_info()[1]
-        except tox.exception.InterpreterNotFound:
-            return sys.exc_info()[1]
         try:
             self.hook.tox_testenv_install_deps(action=action, venv=self)
         except tox.exception.InvocationError:


Test patch:
diff --git a/tests/test_venv.py b/tests/test_venv.py
index c2ca3e46..073c96c3 100644
--- a/tests/test_venv.py
+++ b/tests/test_venv.py
@@ -385,11 +385,11 @@ def test_install_command_not_installed_bash(newmocksession):
 
 
 def test_install_python3(tmpdir, newmocksession):
-    if not py.path.local.sysfind('python3.5'):
-        pytest.skip("needs python3.5")
+    if not py.path.local.sysfind('python3'):
+        pytest.skip("needs python3")
     mocksession = newmocksession([], """
         [testenv:py123]
-        basepython=python3.5
+        basepython=python3
         deps=
             dep1
             dep2
diff --git a/tests/test_z_cmdline.py b/tests/test_z_cmdline.py
index 970a7b64..6ef66d8f 100644
--- a/tests/test_z_cmdline.py
+++ b/tests/test_z_cmdline.py
@@ -312,6 +312,27 @@ def test_skip_unknown_interpreter(cmd, initproj):
     assert any(msg == l for l in result.outlines)
 
 
+def test_skip_unknown_interpreter_result_json(cmd, initproj, tmpdir):
+    report_path = tmpdir.join("toxresult.json")
+    initproj("interp123-0.5", filedefs={
+        'tests': {'test_hello.py': "def test_hello(): pass"},
+        'tox.ini': '''
+            [testenv:python]
+            basepython=xyz_unknown_interpreter
+            [testenv]
+            changedir=tests
+        '''
+    })
+    result = cmd("--skip-missing-interpreters", "--result-json", report_path)
+    assert not result.ret
+    msg = 'SKIPPED:  python: InterpreterNotFound: xyz_unknown_interpreter'
+    assert any(msg == l for l in result.outlines)
+    setup_result_from_json = json.load(report_path)["testenvs"]["python"]["setup"]
+    for setup_step in setup_result_from_json:
+        assert "InterpreterNotFound" in setup_step["output"]
+        assert setup_step["retcode"] == "0"
+
+
 def test_unknown_dep(cmd, initproj):
     initproj("dep123-0.7", filedefs={
         'tests': {'test_hello.py': "def test_hello(): pass"},