Repository: conda/conda-build

Issue:
Conda build with python 3.5 crashing
### Actual Behavior

When I attempt to `conda-build` a package based on `python=3.5` I get errors. The stack trace is

<details>

```
DEBUG:conda_build.index:/home/matthew/anaconda3/envs/test-environment5/conda-bld/linux-64/travis-tester-0.0.11-py35_0.tar.bz2 has no file info/run_exports.yaml
Traceback (most recent call last):
  File "/home/matthew/anaconda3/envs/test-environment5/bin/conda-build", line 11, in <module>
    sys.exit(main())
  File "/home/matthew/anaconda3/envs/test-environment5/lib/python3.5/site-packages/conda_build/cli/main_build.py", line 424, in main
    execute(sys.argv[1:])
  File "/home/matthew/anaconda3/envs/test-environment5/lib/python3.5/site-packages/conda_build/cli/main_build.py", line 415, in execute
    verify=args.verify)
  File "/home/matthew/anaconda3/envs/test-environment5/lib/python3.5/site-packages/conda_build/api.py", line 201, in build
    notest=notest, need_source_download=need_source_download, variants=variants)
  File "/home/matthew/anaconda3/envs/test-environment5/lib/python3.5/site-packages/conda_build/build.py", line 2190, in build_tree
    notest=notest,
  File "/home/matthew/anaconda3/envs/test-environment5/lib/python3.5/site-packages/conda_build/build.py", line 1195, in build
    output_metas = expand_outputs([(m, need_source_download, need_reparse_in_env)])
  File "/home/matthew/anaconda3/envs/test-environment5/lib/python3.5/site-packages/conda_build/render.py", line 679, in expand_outputs
    for (output_dict, m) in _m.get_output_metadata_set(permit_unsatisfiable_variants=False):
  File "/home/matthew/anaconda3/envs/test-environment5/lib/python3.5/site-packages/conda_build/metadata.py", line 1990, in get_output_metadata_set
    bypass_env_check=bypass_env_check)
  File "/home/matthew/anaconda3/envs/test-environment5/lib/python3.5/site-packages/conda_build/metadata.py", line 714, in finalize_outputs_pass
    permit_unsatisfiable_variants=permit_unsatisfiable_variants)
  File "/home/matthew/anaconda3/envs/test-environment5/lib/python3.5/site-packages/conda_build/render.py", line 468, in finalize_metadata
    exclude_pattern)
  File "/home/matthew/anaconda3/envs/test-environment5/lib/python3.5/site-packages/conda_build/render.py", line 341, in add_upstream_pins
    permit_unsatisfiable_variants, exclude_pattern)
  File "/home/matthew/anaconda3/envs/test-environment5/lib/python3.5/site-packages/conda_build/render.py", line 329, in _read_upstream_pin_files
    permit_unsatisfiable_variants=permit_unsatisfiable_variants)
  File "/home/matthew/anaconda3/envs/test-environment5/lib/python3.5/site-packages/conda_build/render.py", line 125, in get_env_dependencies
    channel_urls=tuple(m.config.channel_urls))
  File "/home/matthew/anaconda3/envs/test-environment5/lib/python3.5/site-packages/conda_build/environ.py", line 733, in get_install_actions
    locking=locking, timeout=timeout)
  File "/home/matthew/anaconda3/envs/test-environment5/lib/python3.5/site-packages/conda_build/index.py", line 202, in get_build_index
    update_index(output_folder, verbose=verbose)
  File "/home/matthew/anaconda3/envs/test-environment5/lib/python3.5/site-packages/conda_build/index.py", line 267, in update_index
    return ChannelIndex(dir_path, channel_name, deep_integrity_check=check_md5, threads=threads).index(patch_generator=patch_generator, verbose=verbose)
  File "/home/matthew/anaconda3/envs/test-environment5/lib/python3.5/site-packages/conda_build/index.py", line 749, in index
    repodata_from_packages[subdir] = self.index_subdir(subdir)
  File "/home/matthew/anaconda3/envs/test-environment5/lib/python3.5/site-packages/conda_build/index.py", line 869, in index_subdir
    fn, mtime, size, index_json = future.result()
  File "/home/matthew/anaconda3/envs/test-environment5/lib/python3.5/concurrent/futures/_base.py", line 398, in result
    return self.__get_result()
  File "/home/matthew/anaconda3/envs/test-environment5/lib/python3.5/concurrent/futures/_base.py", line 357, in __get_result
    raise self._exception
  File "/home/matthew/anaconda3/envs/test-environment5/lib/python3.5/concurrent/futures/thread.py", line 55, in run
    result = self.fn(*self.args, **self.kwargs)
  File "/home/matthew/anaconda3/envs/test-environment5/lib/python3.5/site-packages/conda_build/index.py", line 937, in _extract_to_cache
    _cache_post_install_details(binary_paths_json, all_paths, post_install_cache_path)
  File "/home/matthew/anaconda3/envs/test-environment5/lib/python3.5/site-packages/conda_build/index.py", line 592, in _cache_post_install_details
    for f in json.loads(loaded_json_text).get('paths', []):
  File "/home/matthew/anaconda3/envs/test-environment5/lib/python3.5/json/__init__.py", line 312, in loads
    s.__class__.__name__))
TypeError: the JSON object must be str, not 'bytes'
```

</details>

### Expected Behavior

I expect this to build, for example this works fine with

```
conda create -y -q -n test-environment4 python=3.4 conda-build anaconda-client conda-verify
source activate test-environment4
conda-build --no-anaconda-upload .
```

<details>

```
     active environment : test-environment4
    active env location : /home/matthew/anaconda3/envs/test-environment4
            shell level : 2
       user config file : /home/matthew/.condarc
 populated config files : 
          conda version : 4.5.11
    conda-build version : 3.13.0
         python version : 3.6.5.final.0
       base environment : /home/matthew/anaconda3  (writable)
           channel URLs : https://repo.anaconda.com/pkgs/main/linux-64
                          https://repo.anaconda.com/pkgs/main/noarch
                          https://repo.anaconda.com/pkgs/free/linux-64
                          https://repo.anaconda.com/pkgs/free/noarch
                          https://repo.anaconda.com/pkgs/r/linux-64
                          https://repo.anaconda.com/pkgs/r/noarch
                          https://repo.anaconda.com/pkgs/pro/linux-64
                          https://repo.anaconda.com/pkgs/pro/noarch
          package cache : /home/matthew/anaconda3/pkgs
                          /home/matthew/.conda/pkgs
       envs directories : /home/matthew/anaconda3/envs
                          /home/matthew/.conda/envs
               platform : linux-64
             user-agent : conda/4.5.11 requests/2.18.4 CPython/3.6.5 Linux/4.15.0-33-generic ubuntu/18.04 glibc/2.27
                UID:GID : 1000:1000
             netrc file : None
           offline mode : False
```

</details>

### Steps to Reproduce
```
conda create -y -q -n test-environment5 python=3.5 conda-build anaconda-client conda-verify
source activate test-environment5
conda-build --no-anaconda-upload .
```
The associated `meta.yml` file is
```
package:
    name: simple
    version: "0.0.1"

source:
    git_url: ../

requirements:
  host:
    - python
    - pip
  run:
    - python
```
with setup.py

```python
from setuptools import setup


setup(name='simple',
      version="0.0.1",
      description='Testing git and deployment stuff',
      long_description="Nothing",
      license='MIT',
      platforms='any',
      packages=['mypackage'],
      zip_safe=False)
```

This is based on running in a simple project of the form
```
conda_test
  conda.recipe/
    bld.bat
    build.sh
    meta.yaml
  .git/
  mypackage/
    A.py
  setup.py
```

##### `Output of conda info`

```
     active environment : test-environment5
    active env location : /home/matthew/anaconda3/envs/test-environment5
            shell level : 2
       user config file : /home/matthew/.condarc
 populated config files : 
          conda version : 4.5.11
    conda-build version : 3.13.0
         python version : 3.6.5.final.0
       base environment : /home/matthew/anaconda3  (writable)
           channel URLs : https://repo.anaconda.com/pkgs/main/linux-64
                          https://repo.anaconda.com/pkgs/main/noarch
                          https://repo.anaconda.com/pkgs/free/linux-64
                          https://repo.anaconda.com/pkgs/free/noarch
                          https://repo.anaconda.com/pkgs/r/linux-64
                          https://repo.anaconda.com/pkgs/r/noarch
                          https://repo.anaconda.com/pkgs/pro/linux-64
                          https://repo.anaconda.com/pkgs/pro/noarch
          package cache : /home/matthew/anaconda3/pkgs
                          /home/matthew/.conda/pkgs
       envs directories : /home/matthew/anaconda3/envs
                          /home/matthew/.conda/envs
               platform : linux-64
             user-agent : conda/4.5.11 requests/2.18.4 CPython/3.6.5 Linux/4.15.0-33-generic ubuntu/18.04 glibc/2.27
                UID:GID : 1000:1000
             netrc file : None
           offline mode : False
```




Discussion:
Thanks for the report.  I think this might have to do with your default system encoding.  We haven't seen any trouble with many python 3.5 builds that we've done with that conda-build version.

It is still a bug that we need to fix, but it might be hard to identify exactly how to reproduce this, as I'm pretty certain it depends on the state of your system.  If you can help identify the problem, we'll be able to fix it faster.

What does ``locale`` show?

https://www.cyberciti.biz/faq/how-to-set-locales-i18n-on-a-linux-unix/
Isn't the problem likely to be that conda can only be installed in the base env and therefore conda-build the same?
That might trigger the problem, but the root cause is a dumb Unicode thing.
The problem happens for me in the base env as well, the last working version is: 3.12.1. 

3.13.0 does not work anymore...
On my local machine I get
```
matthew@xps:~$ locale
LANG=en_CA.UTF-8
LANGUAGE=en_CA:en
LC_CTYPE="en_CA.UTF-8"
LC_NUMERIC="en_CA.UTF-8"
LC_TIME="en_CA.UTF-8"
LC_COLLATE="en_CA.UTF-8"
LC_MONETARY="en_CA.UTF-8"
LC_MESSAGES="en_CA.UTF-8"
LC_PAPER="en_CA.UTF-8"
LC_NAME="en_CA.UTF-8"
LC_ADDRESS="en_CA.UTF-8"
LC_TELEPHONE="en_CA.UTF-8"
LC_MEASUREMENT="en_CA.UTF-8"
LC_IDENTIFICATION="en_CA.UTF-8"
LC_ALL=
```

Something worth bringing up and that may make it easier to track down and reproduce is that I originally came across this issue on Travis. A link to a broken build is [here](https://travis-ci.org/matthewgilbert/testing/builds/425689230). `locale` from that job shows
```
$ locale
LANG=en_US.UTF-8
LANGUAGE=
LC_CTYPE="en_US.UTF-8"
LC_NUMERIC="en_US.UTF-8"
LC_TIME="en_US.UTF-8"
LC_COLLATE="en_US.UTF-8"
LC_MONETARY="en_US.UTF-8"
LC_MESSAGES="en_US.UTF-8"
LC_PAPER="en_US.UTF-8"
LC_NAME="en_US.UTF-8"
LC_ADDRESS="en_US.UTF-8"
LC_TELEPHONE="en_US.UTF-8"
LC_MEASUREMENT="en_US.UTF-8"
LC_IDENTIFICATION="en_US.UTF-8"
LC_ALL=en_US.UTF-8
```
I tried unsetting LANGUAGE in a docker container, but could not reproduce the problem.  That was probably an incorrect hunch.  If you have any other ideas on how to help reproduce the issue (preferably with a docker container so I can readily obtain it locally myself), that would be very helpful.
Never mind, I got it to reproduce on a local docker container.  I'll try to fix it soon.
@msarahan I'm afraid there is more of this: 

```
Solving environment: ...working... done
Traceback (most recent call last):
  File "/root/miniconda3/bin/conda-build", line 11, in <module>
    sys.exit(main())
  File "/root/miniconda3/lib/python3.5/site-packages/conda_build/cli/main_build.py", line 424, in main
    execute(sys.argv[1:])
  File "/root/miniconda3/lib/python3.5/site-packages/conda_build/cli/main_build.py", line 415, in execute
    verify=args.verify)
  File "/root/miniconda3/lib/python3.5/site-packages/conda_build/api.py", line 201, in build
    notest=notest, need_source_download=need_source_download, variants=variants)
  File "/root/miniconda3/lib/python3.5/site-packages/conda_build/build.py", line 2205, in build_tree
    notest=notest,
  File "/root/miniconda3/lib/python3.5/site-packages/conda_build/build.py", line 1210, in build
    output_metas = expand_outputs([(m, need_source_download, need_reparse_in_env)])
  File "/root/miniconda3/lib/python3.5/site-packages/conda_build/render.py", line 686, in expand_outputs
    for (output_dict, m) in _m.get_output_metadata_set(permit_unsatisfiable_variants=False):
  File "/root/miniconda3/lib/python3.5/site-packages/conda_build/metadata.py", line 1990, in get_output_metadata_set
    bypass_env_check=bypass_env_check)
  File "/root/miniconda3/lib/python3.5/site-packages/conda_build/metadata.py", line 714, in finalize_outputs_pass
    permit_unsatisfiable_variants=permit_unsatisfiable_variants)
  File "/root/miniconda3/lib/python3.5/site-packages/conda_build/render.py", line 475, in finalize_metadata
    exclude_pattern)
  File "/root/miniconda3/lib/python3.5/site-packages/conda_build/render.py", line 348, in add_upstream_pins
    permit_unsatisfiable_variants, exclude_pattern)
  File "/root/miniconda3/lib/python3.5/site-packages/conda_build/render.py", line 339, in _read_upstream_pin_files
    extra_run_specs = get_upstream_pins(m, actions, env)
  File "/root/miniconda3/lib/python3.5/site-packages/conda_build/render.py", line 327, in get_upstream_pins
    specs = _read_specs_from_package(loc, dist)
  File "/root/miniconda3/lib/python3.5/site-packages/conda/exports.py", line 181, in __call__
    value = self.func(*args, **kw)
  File "/root/miniconda3/lib/python3.5/site-packages/conda_build/render.py", line 228, in _read_specs_from_package
    specs = json.loads(specs_json)
  File "/root/miniconda3/lib/python3.5/json/__init__.py", line 312, in loads
    s.__class__.__name__))
TypeError: the JSON object must be str, not 'bytes'
```

With 3.14.4 on python 3.5/linux-64
There is even another :/

```
Traceback (most recent call last):
  File "/root/miniconda3/bin/build-tool", line 11, in <module>
    load_entry_point('build-tool==1.0.0', 'console_scripts', 'build-tool')()
  File "/root/miniconda3/lib/python3.5/site-packages/build_tool-1.0.0-py3.5.egg/build_tool/cli.py", line 39, in main
  File "/root/miniconda3/lib/python3.5/site-packages/build_tool-1.0.0-py3.5.egg/build_tool/build.py", line 60, in build
  File "/root/miniconda3/lib/python3.5/site-packages/conda_build/cli/main_build.py", line 415, in execute
    verify=args.verify)
  File "/root/miniconda3/lib/python3.5/site-packages/conda_build/api.py", line 201, in build
    notest=notest, need_source_download=need_source_download, variants=variants)
  File "/root/miniconda3/lib/python3.5/site-packages/conda_build/build.py", line 2205, in build_tree
    notest=notest,
  File "/root/miniconda3/lib/python3.5/site-packages/conda_build/build.py", line 1599, in build
    built_package = bundlers[output_d.get('type', 'conda')](output_d, m, env, stats)
  File "/root/miniconda3/lib/python3.5/site-packages/conda_build/build.py", line 1030, in bundle_conda
    update_index(os.path.dirname(output_folder), verbose=metadata.config.verbose)
  File "/root/miniconda3/lib/python3.5/site-packages/conda_build/index.py", line 270, in update_index
    patch_generator=patch_generator, verbose=verbose, progress=progress)
  File "/root/miniconda3/lib/python3.5/site-packages/conda_build/index.py", line 760, in index
    repodata_from_packages[subdir] = self.index_subdir(subdir, verbose=verbose, progress=progress)
  File "/root/miniconda3/lib/python3.5/site-packages/conda_build/index.py", line 882, in index_subdir
    fn, mtime, size, index_json = future.result()
  File "/root/miniconda3/lib/python3.5/concurrent/futures/_base.py", line 398, in result
    return self.__get_result()
  File "/root/miniconda3/lib/python3.5/concurrent/futures/_base.py", line 357, in __get_result
    raise self._exception
  File "/root/miniconda3/lib/python3.5/concurrent/futures/thread.py", line 55, in run
    result = self.fn(*self.args, **self.kwargs)
  File "/root/miniconda3/lib/python3.5/site-packages/conda_build/index.py", line 949, in _extract_to_cache
    _cache_run_exports(tf, tar_path, run_exports_cache_path)
  File "/root/miniconda3/lib/python3.5/site-packages/conda_build/index.py", line 666, in _cache_run_exports
    run_exports = json.load(binary_run_exports)
  File "/root/miniconda3/lib/python3.5/json/__init__.py", line 268, in load
    parse_constant=parse_constant, object_pairs_hook=object_pairs_hook, **kw)
  File "/root/miniconda3/lib/python3.5/json/__init__.py", line 312, in loads
    s.__class__.__name__))
TypeError: the JSON object must be str, not 'bytes'
```
I don't have time right now to revisit this. Please consider a PR, since
you have identified where the issues are and presumably my earlier attempt
shows a way to fix these.

On Wed, Sep 12, 2018, 05:00 gabm <notifications@github.com> wrote:

> There is even another :/
>
> Traceback (most recent call last):
>   File "/root/miniconda3/bin/build-tool", line 11, in <module>
>     load_entry_point('build-tool==1.0.0', 'console_scripts', 'build-tool')()
>   File "/root/miniconda3/lib/python3.5/site-packages/build_tool-1.0.0-py3.5.egg/build_tool/cli.py", line 39, in main
>   File "/root/miniconda3/lib/python3.5/site-packages/build_tool-1.0.0-py3.5.egg/build_tool/build.py", line 60, in build
>   File "/root/miniconda3/lib/python3.5/site-packages/conda_build/cli/main_build.py", line 415, in execute
>     verify=args.verify)
>   File "/root/miniconda3/lib/python3.5/site-packages/conda_build/api.py", line 201, in build
>     notest=notest, need_source_download=need_source_download, variants=variants)
>   File "/root/miniconda3/lib/python3.5/site-packages/conda_build/build.py", line 2205, in build_tree
>     notest=notest,
>   File "/root/miniconda3/lib/python3.5/site-packages/conda_build/build.py", line 1599, in build
>     built_package = bundlers[output_d.get('type', 'conda')](output_d, m, env, stats)
>   File "/root/miniconda3/lib/python3.5/site-packages/conda_build/build.py", line 1030, in bundle_conda
>     update_index(os.path.dirname(output_folder), verbose=metadata.config.verbose)
>   File "/root/miniconda3/lib/python3.5/site-packages/conda_build/index.py", line 270, in update_index
>     patch_generator=patch_generator, verbose=verbose, progress=progress)
>   File "/root/miniconda3/lib/python3.5/site-packages/conda_build/index.py", line 760, in index
>     repodata_from_packages[subdir] = self.index_subdir(subdir, verbose=verbose, progress=progress)
>   File "/root/miniconda3/lib/python3.5/site-packages/conda_build/index.py", line 882, in index_subdir
>     fn, mtime, size, index_json = future.result()
>   File "/root/miniconda3/lib/python3.5/concurrent/futures/_base.py", line 398, in result
>     return self.__get_result()
>   File "/root/miniconda3/lib/python3.5/concurrent/futures/_base.py", line 357, in __get_result
>     raise self._exception
>   File "/root/miniconda3/lib/python3.5/concurrent/futures/thread.py", line 55, in run
>     result = self.fn(*self.args, **self.kwargs)
>   File "/root/miniconda3/lib/python3.5/site-packages/conda_build/index.py", line 949, in _extract_to_cache
>     _cache_run_exports(tf, tar_path, run_exports_cache_path)
>   File "/root/miniconda3/lib/python3.5/site-packages/conda_build/index.py", line 666, in _cache_run_exports
>     run_exports = json.load(binary_run_exports)
>   File "/root/miniconda3/lib/python3.5/json/__init__.py", line 268, in load
>     parse_constant=parse_constant, object_pairs_hook=object_pairs_hook, **kw)
>   File "/root/miniconda3/lib/python3.5/json/__init__.py", line 312, in loads
>     s.__class__.__name__))
> TypeError: the JSON object must be str, not 'bytes'
>
> —
> You are receiving this because you were mentioned.
> Reply to this email directly, view it on GitHub
> <https://github.com/conda/conda-build/issues/3119#issuecomment-420588141>,
> or mute the thread
> <https://github.com/notifications/unsubscribe-auth/AACV-V_wOcgo2Xmso5xNlowaZvlM5vw7ks5uaNs8gaJpZM4Wdohh>
> .
>


Resolving patch:
diff --git a/conda_build/index.py b/conda_build/index.py
--- a/conda_build/index.py
+++ b/conda_build/index.py
@@ -663,6 +663,8 @@ def _cache_about_json(tf, tar_path, about_cache_path):
 def _cache_run_exports(tf, tar_path, run_exports_cache_path):
     try:
         binary_run_exports = tf.extractfile('info/run_exports.json')
+        if hasattr(binary_run_exports, "decode"):
+            binary_run_exports = binary_run_exports.decode("utf-8")
         run_exports = json.load(binary_run_exports)
     except KeyError:
         try:
diff --git a/conda_build/render.py b/conda_build/render.py
--- a/conda_build/render.py
+++ b/conda_build/render.py
@@ -224,6 +224,9 @@ def _read_specs_from_package(pkg_loc, pkg_dist):
         # switching to json for consistency in conda-build 4
         specs_yaml = utils.package_has_file(pkg_loc, 'info/run_exports.yaml')
         specs_json = utils.package_has_file(pkg_loc, 'info/run_exports.json')
+        if hasattr(specs_json, "decode"):
+            specs_json = specs_json.decode("utf-8")
+        
         if specs_json:
             specs = json.loads(specs_json)
         elif specs_yaml:


Test patch: