Repository: conda/conda

Issue:
Invalid JSON output
When installing `Jupyter` I sometimes see the following error:

```
post-link :: /etc/machine-id not found ..
bus post-link :: .. using /proc/sys/kernel/random/boot_id
```

When installing with the `--json` flag the error output causes the json to be invalid.  Example:

```
root@head:~# conda create -n test_env2 python jupyter -y --json -q
dbus post-link :: /etc/machine-id not found ..
dbus post-link :: .. using /proc/sys/kernel/random/boot_id
{
  "actions": {
    "LINK": [
      "expat-2.1.0-0 1",
      ...
    ],
    "PREFIX": "/opt/a/b/c/muunitnoc/anaconda/envs/test_env2",
    "SYMLINK_CONDA": [
      "/opt/a/b/c/muunitnoc/anaconda"
    ],
    "op_order": [
      "RM_FETCHED",
      "FETCH",
      "RM_EXTRACTED",
      "EXTRACT",
      "UNLINK",
      "LINK",
      "SYMLINK_CONDA"
    ]
  },
  "success": true
}
```

In my opinion this is a fairly critical -- I need to depend on valid JSON output

cc @kalefranz @koverholt @mingwandroid

Invalid JSON output
When installing `Jupyter` I sometimes see the following error:

```
post-link :: /etc/machine-id not found ..
bus post-link :: .. using /proc/sys/kernel/random/boot_id
```

When installing with the `--json` flag the error output causes the json to be invalid.  Example:

```
root@head:~# conda create -n test_env2 python jupyter -y --json -q
dbus post-link :: /etc/machine-id not found ..
dbus post-link :: .. using /proc/sys/kernel/random/boot_id
{
  "actions": {
    "LINK": [
      "expat-2.1.0-0 1",
      ...
    ],
    "PREFIX": "/opt/a/b/c/muunitnoc/anaconda/envs/test_env2",
    "SYMLINK_CONDA": [
      "/opt/a/b/c/muunitnoc/anaconda"
    ],
    "op_order": [
      "RM_FETCHED",
      "FETCH",
      "RM_EXTRACTED",
      "EXTRACT",
      "UNLINK",
      "LINK",
      "SYMLINK_CONDA"
    ]
  },
  "success": true
}
```

In my opinion this is a fairly critical -- I need to depend on valid JSON output

cc @kalefranz @koverholt @mingwandroid



Discussion:
Which jupyter package and recipe?  Need to see the post-link scripts...

> jupyter 4.2.0  py27_0 defaults

@mingwandroid mentioned that this was a warning, which seems fine as long as the output can be valid JSON

I don't see a jupyter 4.2.0 at http://repo.continuum.io/pkgs/free/linux-64/

Can you give me the link to the package?

The first thing I need to verify is that this isn't something dumb that jupyter is doing with their post-link scripts.  I'm not aware of conda itself ever writing output like `dbus post-link :: /etc/machine-id not found ..`.  My first guess write now is that jupyter pre-/post-link scripts are writing directly to stdout, in violation of the rules for these scripts: http://conda.pydata.org/docs/building/build-scripts.html

I made a mistake:

> jupyter 1.0.0                    py27_3  defaults

http://repo.continuum.io/pkgs/free/linux-64/jupyter-1.0.0-py27_3.tar.bz2

apologies

https://github.com/ContinuumIO/anaconda-recipes/blob/master/dbus/post-link.sh.Linux

Oh, it's doing it correctly.  Yes, this could be a problem that I can fix within conda.

So what should we do here?  If follks put `echo` statements in in post-links is there anything conda can do to capture and output as JSON?

Here's the offending line: https://github.com/conda/conda/blob/4.2.8/conda/install.py#L1049

Would a fix for you be to write to `stderr` if the `--json` flag is passed?

That would definitely work!

Which jupyter package and recipe?  Need to see the post-link scripts...

> jupyter 4.2.0  py27_0 defaults

@mingwandroid mentioned that this was a warning, which seems fine as long as the output can be valid JSON

I don't see a jupyter 4.2.0 at http://repo.continuum.io/pkgs/free/linux-64/

Can you give me the link to the package?

The first thing I need to verify is that this isn't something dumb that jupyter is doing with their post-link scripts.  I'm not aware of conda itself ever writing output like `dbus post-link :: /etc/machine-id not found ..`.  My first guess write now is that jupyter pre-/post-link scripts are writing directly to stdout, in violation of the rules for these scripts: http://conda.pydata.org/docs/building/build-scripts.html

I made a mistake:

> jupyter 1.0.0                    py27_3  defaults

http://repo.continuum.io/pkgs/free/linux-64/jupyter-1.0.0-py27_3.tar.bz2

apologies

https://github.com/ContinuumIO/anaconda-recipes/blob/master/dbus/post-link.sh.Linux

Oh, it's doing it correctly.  Yes, this could be a problem that I can fix within conda.

So what should we do here?  If follks put `echo` statements in in post-links is there anything conda can do to capture and output as JSON?

Here's the offending line: https://github.com/conda/conda/blob/4.2.8/conda/install.py#L1049

Would a fix for you be to write to `stderr` if the `--json` flag is passed?

That would definitely work!


Resolving patch:
diff --git a/conda/install.py b/conda/install.py
--- a/conda/install.py
+++ b/conda/install.py
@@ -1046,7 +1046,8 @@ def messages(prefix):
     path = join(prefix, '.messages.txt')
     try:
         with open(path) as fi:
-            sys.stdout.write(fi.read())
+            fh = sys.stderr if context.json else sys.stdout
+            fh.write(fi.read())
     except IOError:
         pass
     finally:


Test patch: