Repository: conda/conda-build

Issue:
$SP_DIR env variable doesn't seem to point to the correct Python version
<!--
Thanks for opening an issue! A few things to keep in mind:

- This issue tracker is for issues with or enhancements for conda-build.

For conda issues, you should open an issue at https://github.com/conda/conda/issues
For Anaconda issues, you should open an issue at https://github.com/ContinuumIO/anaconda-issues/issues
For an issue with a particular conda-forge package, you should open an issue on the 
corresponding feedstock: https://github.com/conda-forge
-->

### Actual Behavior

<!-- What actually happens? -->

I used `$SP_DIR` in a conda recipe for a Python 3.6 package noarch and it points to the Python 2.7 site-packages. See:

https://github.com/conda-forge/skijumpdesign-feedstock/pull/6#issuecomment-386649293

```
mv: cannot move `/home/conda/feedstock_root/build_artifacts/skijumpdesign_1525450019672/_h_env_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placeho/static' to `/home/conda/feedstock_root/build_artifacts/skijumpdesign_1525450019672/_h_env_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placeho/lib/python2.7/site-packages/': No such file or directory
```

Build: https://circleci.com/gh/conda-forge/skijumpdesign-feedstock/7?utm_campaign=vcs-integration-link&utm_medium=referral&utm_source=github-build-link

Recipe PR:

https://github.com/conda-forge/skijumpdesign-feedstock/pull/6

### Expected Behavior

<!-- What do you think should happen? -->

`$SP_DIR` should expand to the correct python version's site-packages directory.

### Steps to Reproduce

<!-- Steps to reproduce issue. -->

Expand `$SP_DIR` in a recipe on conda forge.

##### `Output of conda info`
<!-- Between the ticks below, paste the output of 'conda info' -->

See the build info on CircleCI.
```

```




Discussion:
Conda-build is doing what it's told, actually: https://github.com/conda-forge/skijumpdesign-feedstock/blob/master/.ci_support/linux_.yaml

the re-rendering looks wrong, though.  Only linux?

This python constraint is very wrong:

```
requirements:
  build:
    - python >=3.5
```

it is much better to do this instead, and not apply constraints to python in your dependencies:

```
build:
  skip: True  # [py<35]
```

Leaving this issue open, because I'm concerned about the absence of other platforms.
Thanks for the help. I tried your suggestion but the conda forge linter gives:

"noarch packages can't have selectors. If the selectors are necessary, please remove noarch: python"

Note that this is noarch, that's why only circleci should be running.
@msarahan the python constraint method is what we currently have in the docs: https://conda-forge.org/docs/meta.html#building-noarch-packages. Should we change it?

@moorepants does the software have an internal version constraint (eg it uses python 3 only features)?
Having it in `build` and `run` works. This is in `build` only, which does not work.

cc @jjhelmus
The conda-forge gitter channel said that I should use `python>=3.5` in both the run and build sections, not the selector form. So I did that, but I'm hitting the same expansion error reported here. It isn't clear to me how conda forge is using the python 2.7 spec you cite above or what I should do about it.
@CJ-Wright Yes, I'm using some Python 3.5 features (not in 3.4) in the application and it doesn't run on Python 2.
Seeing this issue in https://github.com/conda-forge/cypari2-feedstock/pull/4. conda_build_config.yaml has `python 2.7` in it, but `SP_DIR` in test phase is for 3.6

Resolving patch:
diff --git a/conda_build/environ.py b/conda_build/environ.py
--- a/conda_build/environ.py
+++ b/conda_build/environ.py
@@ -258,7 +258,7 @@ def get_dict(m, prefix=None, for_env=True, skip_build_id=False):
         d.update(meta_vars(m, skip_build_id=skip_build_id))
 
     # system
-    d.update(system_vars(d, m))
+    d.update(system_vars(d, m, prefix))
 
     # features
     d.update({feat.upper(): str(int(value)) for feat, value in
@@ -466,14 +466,14 @@ def get_shlib_ext():
         raise NotImplementedError(sys.platform)
 
 
-def windows_vars(m, get_default):
+def windows_vars(m, get_default, prefix):
     """This is setting variables on a dict that is part of the get_default function"""
     # We have gone for the clang values here.
     win_arch = 'i386' if str(m.config.host_arch) == '32' else 'amd64'
     win_msvc = '19.0.0' if PY3 else '15.0.0'
-    library_prefix = join(m.config.host_prefix, 'Library')
+    library_prefix = join(prefix, 'Library')
     drive, tail = m.config.host_prefix.split(':')
-    get_default('SCRIPTS', join(m.config.host_prefix, 'Scripts'))
+    get_default('SCRIPTS', join(prefix, 'Scripts'))
     get_default('LIBRARY_PREFIX', library_prefix)
     get_default('LIBRARY_BIN', join(library_prefix, 'bin'))
     get_default('LIBRARY_INC', join(library_prefix, 'include'))
@@ -518,15 +518,15 @@ def windows_vars(m, get_default):
             get_default(env_var)
 
 
-def unix_vars(m, get_default):
+def unix_vars(m, get_default, prefix):
     """This is setting variables on a dict that is part of the get_default function"""
     get_default('HOME', 'UNKNOWN')
-    get_default('PKG_CONFIG_PATH', join(m.config.host_prefix, 'lib', 'pkgconfig'))
+    get_default('PKG_CONFIG_PATH', join(prefix, 'lib', 'pkgconfig'))
     get_default('CMAKE_GENERATOR', 'Unix Makefiles')
     get_default('SSL_CERT_FILE')
 
 
-def osx_vars(m, get_default):
+def osx_vars(m, get_default, prefix):
     """This is setting variables on a dict that is part of the get_default function"""
     OSX_ARCH = 'i386' if str(m.config.host_arch) == '32' else 'x86_64'
     # 10.7 install_name_tool -delete_rpath causes broken dylibs, I will revisit this ASAP.
@@ -542,7 +542,7 @@ def _machine_and_architecture():
     return platform.machine(), platform.architecture()
 
 
-def linux_vars(m, get_default):
+def linux_vars(m, get_default, prefix):
     """This is setting variables on a dict that is part of the get_default function"""
     platform_machine, platform_architecture = _machine_and_architecture()
     build_arch = platform_machine
@@ -571,7 +571,7 @@ def linux_vars(m, get_default):
     get_default('QEMU_UNAME')
     get_default('DEJAGNU')
     get_default('DISPLAY')
-    get_default('LD_RUN_PATH', m.config.host_prefix + '/lib')
+    get_default('LD_RUN_PATH', prefix + '/lib')
     get_default('BUILD', build_arch + '-conda_' + build_distro + '-linux-gnu')
 
 
@@ -584,7 +584,7 @@ def set_from_os_or_variant(out_dict, key, variant, default):
 
 
 @memoized
-def system_vars(env_dict, m):
+def system_vars(env_dict, m, prefix):
     d = dict()
     # note the dictionary is passed in here - variables are set in that dict if they are non-null
     get_default = lambda key, default='': set_from_os_or_variant(d, key, m.config.variant, default)
@@ -600,14 +600,14 @@ def system_vars(env_dict, m):
         d = prepend_bin_path(d, m.config.host_prefix)
 
     if sys.platform == 'win32':
-        windows_vars(m, get_default)
+        windows_vars(m, get_default, prefix)
     else:
-        unix_vars(m, get_default)
+        unix_vars(m, get_default, prefix)
 
     if sys.platform == 'darwin':
-        osx_vars(m, get_default)
+        osx_vars(m, get_default, prefix)
     elif sys.platform.startswith('linux'):
-        linux_vars(m, get_default)
+        linux_vars(m, get_default, prefix)
 
     return d
 
diff --git a/conda_build/utils.py b/conda_build/utils.py
--- a/conda_build/utils.py
+++ b/conda_build/utils.py
@@ -801,7 +801,12 @@ def get_stdlib_dir(prefix, py_ver):
     if sys.platform == 'win32':
         lib_dir = os.path.join(prefix, 'Lib')
     else:
-        lib_dir = os.path.join(prefix, 'lib', 'python{}'.format(py_ver))
+        lib_dir = os.path.join(prefix, 'lib')
+        python_folder = glob(os.path.join(lib_dir, 'python?.*'))
+        if python_folder:
+            lib_dir = os.path.join(lib_dir, python_folder[0])
+        else:
+            lib_dir = os.path.join(lib_dir, 'python{}'.format(py_ver))
     return lib_dir
 
 


Test patch:
diff --git a/tests/test-recipes/metadata/_test_test_source_files/conda-recipe/meta.yaml b/tests/test-recipes/metadata/_test_test_source_files/conda-recipe/meta.yaml
--- a/tests/test-recipes/metadata/_test_test_source_files/conda-recipe/meta.yaml
+++ b/tests/test-recipes/metadata/_test_test_source_files/conda-recipe/meta.yaml
@@ -8,7 +8,11 @@ source:
 test:
   source_files:
     - test
+  files:
+    - recipe_file
 
   commands:
     - test -e test/text.txt   # [unix]
+    - test -e recipe_file     # [unix]
     - if not exist test\text.txt exit 1  # [win]
+    - if not exist recipe_file exit 1    # [win]
diff --git a/tests/test-recipes/metadata/_test_test_source_files/conda-recipe/recipe_file b/tests/test-recipes/metadata/_test_test_source_files/conda-recipe/recipe_file
new file mode 100644
diff --git a/tests/test-recipes/metadata/python_env_defined/run_test.py b/tests/test-recipes/metadata/python_env_defined/run_test.py
--- a/tests/test-recipes/metadata/python_env_defined/run_test.py
+++ b/tests/test-recipes/metadata/python_env_defined/run_test.py
@@ -1,4 +1,5 @@
 import os
+import sys
 
 assert os.path.exists(os.getenv('PYTHON'))
 assert os.getenv('PY_VER')
@@ -10,3 +11,6 @@
 assert not os.getenv('PERL'), os.getenv('PERL')
 assert not os.getenv('R'), os.getenv('R')
 assert not os.getenv('LUA'), os.getenv('LUA')
+
+if sys.platform == 'win32':
+    assert os.path.dirname(os.getenv('PYTHON')) == os.path.dirname(os.getenv('LIBRARY_PREFIX'))