Repository: conda/conda-build

Issue:
Incorrect path, cannot create directory
<!--
Thanks for opening an issue! A few things to keep in mind:

- This issue tracker is for issues with or enhancements for conda-build.

For conda issues, you should open an issue at https://github.com/conda/conda/issues
For Anaconda issues, you should open an issue at https://github.com/ContinuumIO/anaconda-issues/issues
For an issue with a particular conda-forge package, you should open an issue on the 
corresponding feedstock: https://github.com/conda-forge
-->

I'm experiencing an error on 3.10.4 which was not present on 3.10.2. I have not tested 3.10.3, so I can't comment on whether that version works.

On 3.10.4, the following occurs a the`install_egg_info` step:
```
changing mode of /home/travis/miniconda/conda-bld/rmg_1526922894154/_build_env/bin/thermoEstimator.py to 755
changing mode of /home/travis/miniconda/conda-bld/rmg_1526922894154/_build_env/bin/databaseTest.py to 755
running install_egg_info
Writing /home/travis/miniconda/conda-bld/rmg_1526922894154/_build_env/lib/python2.7/site-packages/RMG_Py-2.1.9-py2.7.egg-info
cp: cannot create directory ‘/home/travis/miniconda/conda-bld/rmg_1526922894154/_h_env_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placeho/lib/python2.7/site-packages’: No such file or directory
source tree in: /home/travis/miniconda/conda-bld/rmg_1526922894154/work
Traceback (most recent call last):
  File "/home/travis/miniconda/bin/conda-build", line 11, in <module>
    sys.exit(main())
  File "/home/travis/miniconda/lib/python2.7/site-packages/conda_build/cli/main_build.py", line 420, in main
    execute(sys.argv[1:])
  File "/home/travis/miniconda/lib/python2.7/site-packages/conda_build/cli/main_build.py", line 411, in execute
    verify=args.verify)
  File "/home/travis/miniconda/lib/python2.7/site-packages/conda_build/api.py", line 200, in build
    notest=notest, need_source_download=need_source_download, variants=variants)
  File "/home/travis/miniconda/lib/python2.7/site-packages/conda_build/build.py", line 2168, in build_tree
    notest=notest,
  File "/home/travis/miniconda/lib/python2.7/site-packages/conda_build/build.py", line 1408, in build
    utils.check_call_env(cmd, env=env, cwd=src_dir, stats=build_stats)
  File "/home/travis/miniconda/lib/python2.7/site-packages/conda_build/utils.py", line 300, in check_call_env
    return _func_defaulting_env_to_os_environ('call', *popenargs, **kwargs)
  File "/home/travis/miniconda/lib/python2.7/site-packages/conda_build/utils.py", line 280, in _func_defaulting_env_to_os_environ
    raise subprocess.CalledProcessError(proc.returncode, _args)
subprocess.CalledProcessError: Command '['/bin/bash', '-e', '/home/travis/miniconda/conda-bld/rmg_1526922894154/work/conda_build.sh']' returned non-zero exit status 1
```

On 3.10.2, the following occurs instead (and then continues with the rest of the build).
```
changing mode of /home/travis/miniconda/conda-bld/rmg_1526926882611/_h_env_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placeho/bin/thermoEstimator.py to 755
changing mode of /home/travis/miniconda/conda-bld/rmg_1526926882611/_h_env_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placeho/bin/databaseTest.py to 755
running install_egg_info
Writing /home/travis/miniconda/conda-bld/rmg_1526926882611/_h_env_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placeho/lib/python2.7/site-packages/RMG_Py-2.1.9-py2.7.egg-info
source tree in: /home/travis/miniconda/conda-bld/rmg_1526926882611/work
Resource usage statistics from building rmg:
   Process count: 6
   CPU time: Sys=0:00:08.5, User=0:08:59.8
   Memory: 827.8M
   Disk usage: 219.1K
   Time elapsed: 0:10:01.6
Packaging rmg
INFO:conda_build.build:Packaging rmg
Packaging rmg-2.1.9-py27_2
INFO:conda_build.build:Packaging rmg-2.1.9-py27_2
compiling .pyc files...
```

It seems related to the switch from `_h_env_...` to `_build_env` and some path not getting changed. I don't have a good way to reproduce this, besides building RMG-Py, but can provide additional info if needed. It seems to me like this shouldn't be related to the specific build setup for RMG-Py though, so perhaps building other packages could also cause this error.

Travis build links:
- Broken (3.10.4): https://travis-ci.org/mliu49/RMG-build/builds/381772481
- Working (3.10.2): https://travis-ci.org/mliu49/RMG-build/builds/381790265

Output of conda info:

```
$ conda info -a
     active environment : None
       user config file : /home/travis/.condarc
 populated config files : 
          conda version : 4.5.4
    conda-build version : not installed
         python version : 2.7.14.final.0
       base environment : /home/travis/miniconda  (writable)
           channel URLs : https://repo.anaconda.com/pkgs/main/linux-64
                          https://repo.anaconda.com/pkgs/main/noarch
                          https://repo.anaconda.com/pkgs/free/linux-64
                          https://repo.anaconda.com/pkgs/free/noarch
                          https://repo.anaconda.com/pkgs/r/linux-64
                          https://repo.anaconda.com/pkgs/r/noarch
                          https://repo.anaconda.com/pkgs/pro/linux-64
                          https://repo.anaconda.com/pkgs/pro/noarch
          package cache : /home/travis/miniconda/pkgs
                          /home/travis/.conda/pkgs
       envs directories : /home/travis/miniconda/envs
                          /home/travis/.conda/envs
               platform : linux-64
             user-agent : conda/4.5.4 requests/2.18.4 CPython/2.7.14 Linux/4.14.12-041412-generic ubuntu/14.04 glibc/2.19
                UID:GID : 2000:2000
             netrc file : None
           offline mode : False
```
The travis build script installs conda-build after checking conda info, so its version is not listed here.



Discussion:
What does your recipe look like?  3.10.4 tried to straighten out some annoying implicit behavior around when the build and host section get split.  You're probably breaking because of it.  #2896 

There might be a fix here, but it'll probably be easier for you to update your recipe to use build/host properly.  If you post your recipe here, I can give you tips.
Oh, I didn't know about the host section. Any tips would be greatly appreciated!

meta.yaml
```
# For conda build
package:
  name: rmg
  version: {{ environ.get('GIT_DESCRIBE_TAG', '') }}

source:
  path: .

build:
  number: {{ environ.get('GIT_DESCRIBE_NUMBER', 0) }}

requirements:
  build:
    - boost ==1.56.0
    - cython >=0.25.2
    - gcc_linux-64 # [linux]
    - clang_osx-64 # [osx]
    - jinja2
    - lpsolve55
    - numpy
    - openbabel
    - pydas >=1.0.2
    - pydqed >=1.0.1
    - pyrdl
    - python
    - pyzmq
    - quantities
    - rdkit >=2015.09.2
    - scipy
    - scoop
    - setuptools

  run:
    - argparse # [py26]
    - cairo # [unix]
    - cairocffi # [unix]
    - cantera >=2.3.0a3
    - coolprop
    - coverage
    - cython >=0.25.2
    - ffmpeg
    - gprof2dot
    - graphviz
    - guppy
    - jinja2
    - lpsolve55
    - markupsafe
    - matplotlib >=1.5
    - mock
    - mopac
    - nose
    - numpy
    - openbabel
    - psutil
    - pydas >=1.0.2
    - pydot ==1.2.2
    - pydqed >=1.0.1
    - pymongo
    - pyparsing
    - pyrdl
    - python
    - pyzmq
    - quantities
    - rdkit >=2015.09.2
    - rmgdatabase >=2.1.9
    - scipy
    - scoop
    - symmetry
    - xlwt

test:
  source_files:
    - 'examples/rmg/minimal'
  imports:
    - rmgpy
  commands:
    - rmg.py examples/rmg/minimal/input.py

about:
  home: http://github.com/ReactionMechanismGenerator/RMG-Py
  license: MIT
  summary: "A program for automatically generating kinetic models of chemical reaction mechanisms."
```

build.sh
```
make install
cp -R ${SRC_DIR}/external ${SP_DIR}
```

You're using the compiler packages manually, but you should be using the jinja2 functions.  These have strong run_exports, and that's the reason for the breakage to you.  I highly advise you to read the blog post at https://www.anaconda.com/blog/developer-blog/utilizing-the-new-compilers-in-anaconda-distribution-5/ and the docs linked therein.  This section may also be helpful to you: https://conda.io/docs/user-guide/tasks/build-packages/define-metadata.html#host

Your requirements should look like:

```
requirements:
  build:
    - {{ compiler('c') }}
  host:
    - boost ==1.56.0
    - cython >=0.25.2
    - jinja2
    - lpsolve55
    - numpy
    - openbabel
    - pydas >=1.0.2
    - pydqed >=1.0.1
    - pyrdl
    - python
    - pyzmq
    - quantities
    - rdkit >=2015.09.2
    - scipy
    - scoop
    - setuptools

  run:
    - argparse # [py26]
    - cairo # [unix]
    - cairocffi # [unix]
    - cantera >=2.3.0a3
    - coolprop
    - coverage
    - cython >=0.25.2
    - ffmpeg
    - gprof2dot
    - graphviz
    - guppy
    - jinja2
    - lpsolve55
    - markupsafe
    - matplotlib >=1.5
    - mock
    - mopac
    - nose
    - {{ pin_compatible('numpy') }}
    - openbabel
    - psutil
    - pydas >=1.0.2
    - pydot ==1.2.2
    - pydqed >=1.0.1
    - pymongo
    - pyparsing
    - pyrdl
    - python
    - pyzmq
    - quantities
    - rdkit >=2015.09.2
    - rmgdatabase >=2.1.9
    - scipy
    - scoop
    - symmetry
    - xlwt
```

I added in pin_compatible('numpy') just on a wild guess that you compile against numpy's C interface.  If you know that you don't, you can revert that back to plain numpy.
Many thanks! I'll definitely read through the links you provided.
When using conda-build 3.10.4 if I print the environment variables via `printenv` in `run_rest.sh` I got:

`SP_DIR=/home/user/project/env/conda-bld/project_1526982554741/_h_env_placehold_placehold_placehold_placehol/lib/python3.6/site-packages`

But the directory pointed by SP_DIR does not exist.

Also the following environment variables are pointing to the inexisting "_h_env_..." dir:
* LD_RUN_PATH
* STDLIB_DIR
* PKG_CONFIG_PATH
* PYTHON

This issue was introduced by:
https://github.com/conda/conda-build/pull/2896

To reproduce this and related issues you can generat a new app with:
https://github.com/tecnickcom/pygen

Downgrading conda-build to 3.10.2 fixed the issue.
@nicolaasuni that SP_DIR looks abridged.  The total path length of /home/user/project/env/conda-bld/project_1526982554741/_h_env_placehold_placehold_placehold_placehol should be padded to 255 chars.

The host env is the place where things are looked for for packaging purposes.  The build env only exists when it has something that the host env does not.  #2896 made that delineation more well-defined, but I don't see why it would break your recipes, since it looks like you are defining the host section properly.

I don't have time to figure out your tool, but if you want to post an minimal reproducing recipe as a gist on github, or paste it here, I can look at it.
@msarahan Thanks for your prompt answer.

The SP_DIR example I have posted has been redacted to hide the real directory names, that's why is not 255 chars long.

My Conda build process is breaking at the test phase when the "run_test.sh" script is invoked.

When executing the "run_test.sh" script the following environmental variables set by Conda are all pointing to a missing directory "_h_env_..."
* SP_DIR
* LD_RUN_PATH
* STDLIB_DIR
* PKG_CONFIG_PATH
* PYTHON

You should be able to reproduce the issue by adding the "printenv" command in a "run_tests.sh" conda script and look at the printed values.

Even a simple run_test.sh script like:
```
echo -e "\n\n*** TEST ***\n\n"
cd ../work/python && ${PYTHON} setup.py test
```
breakes because the PYTHON variable is pointing to the wrong place.


To build a dummy project and reproduce the error yourself:

```
cd /tmp
git clone git@github.com:tecnickcom/pygen.git
cd pygen
make new TYPE=lib
cd target/github.com/dummyvendor/dummy
make build
{code}

You should get the following output:

{code}
make build
rm -rf target Dockerfile htmlcov build dist .cache .benchmarks ./tests/*.so ./tests/__pycache__ ./dummy/__pycache__ ./dummy.egg-info
find . -type f -name '*.pyc' -exec rm -f {} \;
sed -i "s/version:.*$/version: 1.0.0/" conda/meta.yaml
sed -i "s/number:.*$/number: 1/" conda/meta.yaml
sed -i "s/__version__.*$/__version__ = '1.0.0'/" dummy/__init__.py
sed -i "s/__release__.*$/__release__ = '1'/" dummy/__init__.py
./conda/setup-conda.sh
+ : https://repo.continuum.io/miniconda
+ : Miniconda3-4.4.10-Linux-x86_64.sh
+ : -
+ : -
+ : env-dummy
++ command -v greadlink
+ '[' -x '' ']'
+ READLINK=readlink
+++ dirname ./conda/setup-conda.sh
++ readlink -f ./conda
+ PROJECT_CONDA=/tmp/pygen/target/github.com/dummyvendor/dummy/conda
++ dirname /tmp/pygen/target/github.com/dummyvendor/dummy/conda
+ PROJECT_HOME=/tmp/pygen/target/github.com/dummyvendor/dummy
++ dirname /tmp/pygen/target/github.com/dummyvendor/dummy
+ PROJECT_ROOT=/tmp/pygen/target/github.com/dummyvendor
+ PROJECT_ENV=/tmp/pygen/target/github.com/dummyvendor/env-dummy
+ '[' '!' -d /tmp/pygen/target/github.com/dummyvendor/env-dummy ']'
+ mkdir -p /tmp/pygen/target/github.com/dummyvendor/dummy/target
+ curl https://repo.continuum.io/miniconda/Miniconda3-4.4.10-Linux-x86_64.sh
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100 55.6M  100 55.6M    0     0  21.2M      0  0:00:02  0:00:02 --:--:-- 21.2M
+ bash /tmp/pygen/target/github.com/dummyvendor/dummy/target/Miniconda3-4.4.10-Linux-x86_64.sh -b -p /tmp/pygen/target/github.com/dummyvendor/env-dummy
PREFIX=/tmp/pygen/target/github.com/dummyvendor/env-dummy
installing: python-3.6.4-hc3d631a_1 ...
Python 3.6.4 :: Anaconda, Inc.
installing: ca-certificates-2017.08.26-h1d4fec5_0 ...
installing: conda-env-2.6.0-h36134e3_1 ...
installing: libgcc-ng-7.2.0-h7cc24e2_2 ...
installing: libstdcxx-ng-7.2.0-h7a57d05_2 ...
installing: libffi-3.2.1-hd88cf55_4 ...
installing: ncurses-6.0-h9df7e31_2 ...
installing: openssl-1.0.2n-hb7f436b_0 ...
installing: tk-8.6.7-hc745277_3 ...
installing: xz-5.2.3-h55aa19d_2 ...
installing: yaml-0.1.7-had09818_2 ...
installing: zlib-1.2.11-ha838bed_2 ...
installing: libedit-3.1-heed3624_0 ...
installing: readline-7.0-ha6073c6_4 ...
installing: sqlite-3.22.0-h1bed415_0 ...
installing: asn1crypto-0.24.0-py36_0 ...
installing: certifi-2018.1.18-py36_0 ...
installing: chardet-3.0.4-py36h0f667ec_1 ...
installing: idna-2.6-py36h82fb2a8_1 ...
installing: pycosat-0.6.3-py36h0a5515d_0 ...
installing: pycparser-2.18-py36hf9f622e_1 ...
installing: pysocks-1.6.7-py36hd97a5b1_1 ...
installing: ruamel_yaml-0.15.35-py36h14c3975_1 ...
installing: six-1.11.0-py36h372c433_1 ...
installing: cffi-1.11.4-py36h9745a5d_0 ...
installing: setuptools-38.4.0-py36_0 ...
installing: cryptography-2.1.4-py36hd09be54_0 ...
installing: wheel-0.30.0-py36hfd4bba0_1 ...
installing: pip-9.0.1-py36h6c6f9ce_4 ...
installing: pyopenssl-17.5.0-py36h20ba746_0 ...
installing: urllib3-1.22-py36hbe7ace6_0 ...
installing: requests-2.18.4-py36he2e5f8d_1 ...
installing: conda-4.4.10-py36_0 ...
installation finished.
+ '[' - '!=' - ']'
+ '[' - '!=' - ']'
+ set +u
+ /tmp/pygen/target/github.com/dummyvendor/env-dummy/bin/conda install -y conda-build pyyaml
Solving environment: done

## Package Plan ##

  environment location: /tmp/pygen/target/github.com/dummyvendor/env-dummy

  added / updated specs: 
    - conda-build
    - pyyaml


The following packages will be downloaded:

    package                    |            build
    ---------------------------|-----------------
    ca-certificates-2018.03.07 |                0         124 KB
    markupsafe-1.0             |   py36hd9260cd_1          24 KB
    beautifulsoup4-4.6.0       |   py36h49b8c8c_1         133 KB
    glob2-0.6                  |   py36he249c77_0          17 KB
    pyyaml-3.12                |   py36hafb9ca4_1         159 KB
    patchelf-0.9               |       hf79760b_2          69 KB
    conda-verify-2.0.0         |   py36h98955d8_0          20 KB
    psutil-5.4.5               |   py36h14c3975_0         303 KB
    pkginfo-1.4.2              |           py36_1          39 KB
    conda-4.5.4                |           py36_0         1.0 MB
    openssl-1.0.2o             |       h20670df_0         3.4 MB
    filelock-3.0.4             |           py36_0          12 KB
    conda-build-3.10.4         |           py36_0         431 KB
    certifi-2018.4.16          |           py36_0         142 KB
    jinja2-2.10                |   py36ha16c418_0         184 KB
    ------------------------------------------------------------
                                           Total:         6.1 MB

The following NEW packages will be INSTALLED:

    beautifulsoup4:  4.6.0-py36h49b8c8c_1 
    conda-build:     3.10.4-py36_0        
    conda-verify:    2.0.0-py36h98955d8_0 
    filelock:        3.0.4-py36_0         
    glob2:           0.6-py36he249c77_0   
    jinja2:          2.10-py36ha16c418_0  
    markupsafe:      1.0-py36hd9260cd_1   
    patchelf:        0.9-hf79760b_2       
    pkginfo:         1.4.2-py36_1         
    psutil:          5.4.5-py36h14c3975_0 
    pyyaml:          3.12-py36hafb9ca4_1  

The following packages will be UPDATED:

    ca-certificates: 2017.08.26-h1d4fec5_0 --> 2018.03.07-0     
    certifi:         2018.1.18-py36_0      --> 2018.4.16-py36_0 
    conda:           4.4.10-py36_0         --> 4.5.4-py36_0     
    openssl:         1.0.2n-hb7f436b_0     --> 1.0.2o-h20670df_0


Downloading and Extracting Packages
ca-certificates 2018.03.07: ####################################################################################################################### | 100% 
markupsafe 1.0: ################################################################################################################################### | 100% 
beautifulsoup4 4.6.0: ############################################################################################################################# | 100% 
glob2 0.6: ######################################################################################################################################## | 100% 
pyyaml 3.12: ###################################################################################################################################### | 100% 
patchelf 0.9: ##################################################################################################################################### | 100% 
conda-verify 2.0.0: ############################################################################################################################### | 100% 
psutil 5.4.5: ##################################################################################################################################### | 100% 
pkginfo 1.4.2: #################################################################################################################################### | 100% 
conda 4.5.4: ###################################################################################################################################### | 100% 
openssl 1.0.2o: ################################################################################################################################### | 100% 
filelock 3.0.4: ################################################################################################################################### | 100% 
conda-build 3.10.4: ############################################################################################################################### | 100% 
certifi 2018.4.16: ################################################################################################################################ | 100% 
jinja2 2.10: ###################################################################################################################################### | 100% 
Preparing transaction: done
Verifying transaction: done
Executing transaction: done
mkdir -p target
"/tmp/pygen/target/github.com/dummyvendor/env-dummy/bin/conda" build --prefix-length 128 --no-anaconda-upload --no-remove-work-dir --override-channels -c conda-forge -c https://repo.continuum.io/pkgs/main -c https://repo.continuum.io/pkgs/free conda
Adding in variants from internal_defaults
INFO:conda_build.variants:Adding in variants from internal_defaults
Attempting to finalize metadata for dummy
INFO:conda_build.metadata:Attempting to finalize metadata for dummy
Solving environment: ...working... done
Solving environment: ...working... done
BUILD START: ['dummy-1.0.0-py36h14c3975_1.tar.bz2']
Solving environment: ...working... done

## Package Plan ##

  environment location: /tmp/pygen/target/github.com/dummyvendor/env-dummy/conda-bld/dummy_1526993665671/_h_env_placehold_placehold_placehold_placehold_


The following NEW packages will be INSTALLED:

    ca-certificates: 2018.4.16-0      conda-forge
    certifi:         2018.4.16-py36_0 conda-forge
    ncurses:         5.9-10           conda-forge
    openssl:         1.0.2o-0         conda-forge
    pip:             9.0.3-py36_0     conda-forge
    python:          3.6.5-1          conda-forge
    readline:        7.0-0            conda-forge
    setuptools:      39.2.0-py36_0    conda-forge
    sqlite:          3.20.1-2         conda-forge
    tk:              8.6.7-0          conda-forge
    wheel:           0.31.0-py36_0    conda-forge
    xz:              5.2.3-0          conda-forge
    zlib:            1.2.11-0         conda-forge

Preparing transaction: ...working... done
Verifying transaction: ...working... done
Executing transaction: ...working... done
Solving environment: ...working... done
Copying /tmp/pygen/target/github.com/dummyvendor/dummy to /tmp/pygen/target/github.com/dummyvendor/env-dummy/conda-bld/dummy_1526993665671/work
source tree in: /tmp/pygen/target/github.com/dummyvendor/env-dummy/conda-bld/dummy_1526993665671/work


*** BUILD ***


running install
running build
running build_py
creating build
creating build/lib
creating build/lib/dummy
copying dummy/__init__.py -> build/lib/dummy
copying dummy/process.py -> build/lib/dummy
running install_lib
creating /tmp/pygen/target/github.com/dummyvendor/env-dummy/conda-bld/dummy_1526993665671/_h_env_placehold_placehold_placehold_placehold_/lib/python3.6/site-packages/dummy
copying build/lib/dummy/__init__.py -> /tmp/pygen/target/github.com/dummyvendor/env-dummy/conda-bld/dummy_1526993665671/_h_env_placehold_placehold_placehold_placehold_/lib/python3.6/site-packages/dummy
copying build/lib/dummy/process.py -> /tmp/pygen/target/github.com/dummyvendor/env-dummy/conda-bld/dummy_1526993665671/_h_env_placehold_placehold_placehold_placehold_/lib/python3.6/site-packages/dummy
byte-compiling /tmp/pygen/target/github.com/dummyvendor/env-dummy/conda-bld/dummy_1526993665671/_h_env_placehold_placehold_placehold_placehold_/lib/python3.6/site-packages/dummy/__init__.py to __init__.cpython-36.pyc
byte-compiling /tmp/pygen/target/github.com/dummyvendor/env-dummy/conda-bld/dummy_1526993665671/_h_env_placehold_placehold_placehold_placehold_/lib/python3.6/site-packages/dummy/process.py to process.cpython-36.pyc
running install_data
creating /tmp/pygen/target/github.com/dummyvendor/env-dummy/conda-bld/dummy_1526993665671/_h_env_placehold_placehold_placehold_placehold_/info
copying VERSION -> /tmp/pygen/target/github.com/dummyvendor/env-dummy/conda-bld/dummy_1526993665671/_h_env_placehold_placehold_placehold_placehold_/info
copying RELEASE -> /tmp/pygen/target/github.com/dummyvendor/env-dummy/conda-bld/dummy_1526993665671/_h_env_placehold_placehold_placehold_placehold_/info
copying LICENSE -> /tmp/pygen/target/github.com/dummyvendor/env-dummy/conda-bld/dummy_1526993665671/_h_env_placehold_placehold_placehold_placehold_/info
copying README.md -> /tmp/pygen/target/github.com/dummyvendor/env-dummy/conda-bld/dummy_1526993665671/_h_env_placehold_placehold_placehold_placehold_/info
running install_egg_info
running egg_info
creating dummy.egg-info
writing dummy.egg-info/PKG-INFO
writing dependency_links to dummy.egg-info/dependency_links.txt
writing requirements to dummy.egg-info/requires.txt
writing top-level names to dummy.egg-info/top_level.txt
writing manifest file 'dummy.egg-info/SOURCES.txt'
reading manifest file 'dummy.egg-info/SOURCES.txt'
reading manifest template 'MANIFEST.in'
warning: no previously-included files found matching '.gitignore'
warning: no previously-included files found matching '.coverage'
warning: no previously-included files found matching '.coveragerc'
warning: no previously-included files found matching '.travis.yml'
warning: no files found matching 'setup.cfg'
warning: no files found matching 'resources/etc/dummy/*.json'
no previously-included directories found matching '.cache'
no previously-included directories found matching '.git'
no previously-included directories found matching 'build'
no previously-included directories found matching 'dist'
no previously-included directories found matching 'htmlcov'
writing manifest file 'dummy.egg-info/SOURCES.txt'
Copying dummy.egg-info to /tmp/pygen/target/github.com/dummyvendor/env-dummy/conda-bld/dummy_1526993665671/_h_env_placehold_placehold_placehold_placehold_/lib/python3.6/site-packages/dummy-1.0.0-py3.6.egg-info
running install_scripts
writing list of installed files to 'record.txt'

Resource usage statistics from building dummy:
   Process count: 1
   CPU time: unavailable
   Memory: 3.3M
   Disk usage: 55.8K
   Time elapsed: 0:00:02.1

Packaging dummy
INFO:conda_build.build:Packaging dummy
Packaging dummy-1.0.0-py36h14c3975_1
INFO:conda_build.build:Packaging dummy-1.0.0-py36h14c3975_1
compiling .pyc files...
number of files: 13
Fixing permissions
updating: dummy-1.0.0-py36h14c3975_1.tar.bz2
TEST START: /tmp/pygen/target/github.com/dummyvendor/env-dummy/conda-bld/linux-64/dummy-1.0.0-py36h14c3975_1.tar.bz2
Updating index at /tmp/pygen/target/github.com/dummyvendor/env-dummy/conda-bld/linux-64 to make package installable with dependencies
INFO:conda_build.build:Updating index at /tmp/pygen/target/github.com/dummyvendor/env-dummy/conda-bld/linux-64 to make package installable with dependencies
Updating index at /tmp/pygen/target/github.com/dummyvendor/env-dummy/conda-bld/noarch to make package installable with dependencies
INFO:conda_build.build:Updating index at /tmp/pygen/target/github.com/dummyvendor/env-dummy/conda-bld/noarch to make package installable with dependencies
Not moving work directory after build.  Your package may depend on files in the work directory that are not included with your package
WARNING:conda_build.build:Not moving work directory after build.  Your package may depend on files in the work directory that are not included with your package
Solving environment: ...working... done

## Package Plan ##

  environment location: /tmp/pygen/target/github.com/dummyvendor/env-dummy/conda-bld/dummy_1526993665671/_test_env_placehold_placehold_placehold_placeho


The following NEW packages will be INSTALLED:

    attrs:            18.1.0-py_0          conda-forge
    ca-certificates:  2018.4.16-0          conda-forge
    certifi:          2018.4.16-py36_0     conda-forge
    coverage:         4.5.1-py36_0         conda-forge
    dummy:            1.0.0-py36h14c3975_1 local      
    more-itertools:   4.1.0-py_0           conda-forge
    ncurses:          5.9-10               conda-forge
    openssl:          1.0.2o-0             conda-forge
    pip:              9.0.3-py36_0         conda-forge
    pluggy:           0.6.0-py_0           conda-forge
    py:               1.5.3-py_0           conda-forge
    pycodestyle:      2.4.0-py36_0         conda-forge
    pytest:           3.5.1-py36_0         conda-forge
    pytest-benchmark: 3.0.0-py36_1         conda-forge
    pytest-cov:       2.5.1-py36_0         conda-forge
    python:           3.6.5-1              conda-forge
    readline:         7.0-0                conda-forge
    setuptools:       39.2.0-py36_0        conda-forge
    six:              1.11.0-py36_1        conda-forge
    sqlite:           3.20.1-2             conda-forge
    tk:               8.6.7-0              conda-forge
    wheel:            0.31.0-py36_0        conda-forge
    xz:               5.2.3-0              conda-forge
    zlib:             1.2.11-0             conda-forge

Preparing transaction: ...working... done
Verifying transaction: ...working... done
Executing transaction: ...working... done
import: 'dummy'


*** TEST ***


/tmp/pygen/target/github.com/dummyvendor/env-dummy/conda-bld/dummy_1526993665671/test_tmp/run_test.sh: line 3: /tmp/pygen/target/github.com/dummyvendor/env-dummy/conda-bld/dummy_1526993665671/_h_env_placehold_placehold_placehold_placehold_/lib/python3.6/site-packages/subcov.pth: No such file or directory
Tests failed for dummy-1.0.0-py36h14c3975_1.tar.bz2 - moving package to /tmp/pygen/target/github.com/dummyvendor/env-dummy/conda-bld/broken
WARNING:conda_build.build:Tests failed for dummy-1.0.0-py36h14c3975_1.tar.bz2 - moving package to /tmp/pygen/target/github.com/dummyvendor/env-dummy/conda-bld/broken
removing: dummy-1.0.0-py36h14c3975_1.tar.bz2
TESTS FAILED: dummy-1.0.0-py36h14c3975_1.tar.bz2
Makefile:87: recipe for target 'build' failed
make: *** [build] Error 1
```


Sorry, I somehow missed the "test" part of your issue.  During test time, that variable should point to the _test_env_(padding junk).  That is a bug.  I'll fix it soon.

Resolving patch:
diff --git a/conda_build/build.py b/conda_build/build.py
--- a/conda_build/build.py
+++ b/conda_build/build.py
@@ -2169,7 +2169,7 @@ def build_tree(recipe_list, config, stats, build_only=False, post=False, notest=
                                            )
                 if not notest:
                     for pkg, dict_and_meta in packages_from_this.items():
-                        if pkg.endswith('.tar.bz2'):
+                        if pkg.endswith('.tar.bz2') and os.path.isfile(pkg):
                             # we only know how to test conda packages
                             test(pkg, config=metadata.config.copy(), stats=stats)
                         _, meta = dict_and_meta
diff --git a/conda_build/environ.py b/conda_build/environ.py
--- a/conda_build/environ.py
+++ b/conda_build/environ.py
@@ -249,10 +249,10 @@ def get_dict(m, prefix=None, for_env=True, skip_build_id=False):
     d = conda_build_vars(prefix, m.config)
 
     # languages
-    d.update(python_vars(m))
-    d.update(perl_vars(m))
-    d.update(lua_vars(m))
-    d.update(r_vars(m))
+    d.update(python_vars(m, prefix))
+    d.update(perl_vars(m, prefix))
+    d.update(lua_vars(m, prefix))
+    d.update(r_vars(m, prefix))
 
     if m:
         d.update(meta_vars(m, skip_build_id=skip_build_id))
@@ -301,22 +301,21 @@ def conda_build_vars(prefix, config):
     }
 
 
-def python_vars(metadata):
+def python_vars(metadata, prefix):
     py_ver = get_py_ver(metadata.config)
     vars_ = {
             'CONDA_PY': ''.join(py_ver.split('.')[:2]),
             'PY3K': str(int(int(py_ver[0]) >= 3)),
             'PY_VER': py_ver,
-            'STDLIB_DIR': utils.get_stdlib_dir(metadata.config.host_prefix, py_ver),
-            'SP_DIR': utils.get_site_packages(metadata.config.host_prefix, py_ver),
+            'STDLIB_DIR': utils.get_stdlib_dir(prefix, py_ver),
+            'SP_DIR': utils.get_site_packages(prefix, py_ver),
             }
     build_or_host = 'host' if metadata.is_cross else 'build'
     deps = [str(ms.name) for ms in metadata.ms_depends(build_or_host)]
     if 'python' in deps:
         vars_.update({
             # host prefix is always fine, because it is the same as build when is_cross is False
-            'PYTHON': metadata.config.python_bin(metadata.config.host_prefix,
-                                                 metadata.config.host_subdir),
+            'PYTHON': metadata.config.python_bin(prefix, metadata.config.host_subdir),
         })
 
     np_ver = metadata.config.variant.get('numpy', get_default_variant(metadata.config)['numpy'])
@@ -325,7 +324,7 @@ def python_vars(metadata):
     return vars_
 
 
-def perl_vars(metadata):
+def perl_vars(metadata, prefix):
     vars_ = {
             'PERL_VER': get_perl_ver(metadata.config),
             'CONDA_PERL': get_perl_ver(metadata.config),
@@ -335,13 +334,12 @@ def perl_vars(metadata):
     if 'perl' in deps:
         vars_.update({
             # host prefix is always fine, because it is the same as build when is_cross is False
-            'PERL': metadata.config.perl_bin(metadata.config.host_prefix,
-                                             metadata.config.host_subdir),
+            'PERL': metadata.config.perl_bin(prefix, metadata.config.host_subdir),
         })
     return vars_
 
 
-def lua_vars(metadata):
+def lua_vars(metadata, prefix):
     vars_ = {
             'LUA_VER': get_lua_ver(metadata.config),
             'CONDA_LUA': get_lua_ver(metadata.config),
@@ -350,14 +348,14 @@ def lua_vars(metadata):
     deps = [str(ms.name) for ms in metadata.ms_depends(build_or_host)]
     if 'lua' in deps:
         vars_.update({
-            'LUA': metadata.config.lua_bin(metadata.config.host_prefix,
+            'LUA': metadata.config.lua_bin(prefix,
                                            metadata.config.host_subdir),
             'LUA_INCLUDE_DIR': get_lua_include_dir(metadata.config),
         })
     return vars_
 
 
-def r_vars(metadata):
+def r_vars(metadata, prefix):
     vars_ = {
             'R_VER': get_r_ver(metadata.config),
             'CONDA_R': get_r_ver(metadata.config),
@@ -367,8 +365,7 @@ def r_vars(metadata):
     deps = [str(ms.name) for ms in metadata.ms_depends(build_or_host)]
     if 'r-base' in deps or 'mro-base' in deps:
         vars_.update({
-            'R': metadata.config.r_bin(metadata.config.host_prefix,
-                                       metadata.config.host_subdir),
+            'R': metadata.config.r_bin(prefix, metadata.config.host_subdir),
         })
     return vars_
 


Test patch:
diff --git a/tests/test-recipes/metadata/python_build/build.sh b/tests/test-recipes/metadata/python_build/build.sh
--- a/tests/test-recipes/metadata/python_build/build.sh
+++ b/tests/test-recipes/metadata/python_build/build.sh
@@ -1,9 +1,15 @@
 if [ -z "${STDLIB_DIR:-}" ]; then
     echo "STDLIB_DIR is unset";
     exit 1
+else
+    echo "STDLIB_DIR is set to $STDLIB_DIR";
+    echo "STDLIB_DIR has $(ls -l $STDLIB_DIR | wc -l) lines"
 fi
 
 if [ -z "${SP_DIR:-}" ]; then
     echo "SP_DIR is unset";
     exit 1
+else
+    echo "SP_DIR is set to $SP_DIR";
+    echo "SP_DIR has $(ls -l $SP_DIR | wc -l) lines"
 fi
diff --git a/tests/test-recipes/metadata/python_env_defined/meta.yaml b/tests/test-recipes/metadata/python_env_defined/meta.yaml
--- a/tests/test-recipes/metadata/python_env_defined/meta.yaml
+++ b/tests/test-recipes/metadata/python_env_defined/meta.yaml
@@ -5,3 +5,5 @@ package:
 requirements:
   build:
     - python
+  run:
+    - python
diff --git a/tests/test-recipes/metadata/python_env_defined/run_test.py b/tests/test-recipes/metadata/python_env_defined/run_test.py
--- a/tests/test-recipes/metadata/python_env_defined/run_test.py
+++ b/tests/test-recipes/metadata/python_env_defined/run_test.py
@@ -1,11 +1,11 @@
 import os
 
-assert os.getenv('PYTHON')
+assert os.path.exists(os.getenv('PYTHON'))
 assert os.getenv('PY_VER')
 assert os.getenv('CONDA_PY')
 assert '.' not in os.getenv('CONDA_PY')
-assert os.getenv('STDLIB_DIR')
-assert os.getenv('SP_DIR')
+assert os.path.exists(os.getenv('STDLIB_DIR'))
+assert os.path.exists(os.getenv('SP_DIR'))
 assert os.getenv('PY3K')
 assert not os.getenv('PERL'), os.getenv('PERL')
 assert not os.getenv('R'), os.getenv('R')