Repository: conda/conda-build

Issue:
3.0.5 python x.x use throws exception

💥 

## Relevant example recipe bits:

```yaml
package:
  name: example_pkg
  version: 1.0.0

requirements:
  build:
    - numpy
    - python x.x

  run:
    - numpy
    - python x.x
```

## Fails with

```python
conda.exceptions.NoPackagesFoundError: Package missing in current osx-64 channels:
  - python x.x.*
```

and 

```python
conda_build.exceptions.DependencyNeedsBuildingError: Unsatisfiable dependencies for platform osx-64: ['python']
```

## Full traceback

```python
$ conda build example_pkg --python=3.5 --numpy=1.12

Leaving build/test directories:
  Work:	 ../../../../home/jhunk/anaconda3/conda-bld/work
  Test:	 ../../../../home/jhunk/anaconda3/conda-bld/test_tmp
Leaving build/test environments:
  Test:	source activate  ../../../../home/jhunk/anaconda3/conda-bld/_test_env_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_pla
  Build:	source activate  ../../../../home/jhunk/anaconda3/conda-bld/_h_env_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placeh


Traceback (most recent call last):
  File "/home/jhunk/anaconda3/lib/python3.5/site-packages/conda_build/environ.py", line 700, in get_install_actions
    actions = install_actions(prefix, index, specs, force=True)
  File "/home/jhunk/anaconda3/lib/python3.5/site-packages/conda/plan.py", line 461, in install_actions
    update_deps, pinned)
  File "/home/jhunk/anaconda3/lib/python3.5/site-packages/conda/plan.py", line 634, in get_actions_for_dists
    pkgs = r.install(specs, installed, update_deps=update_deps)
  File "/home/jhunk/anaconda3/lib/python3.5/site-packages/conda/resolve.py", line 811, in install
    pkgs = self.solve(specs, returnall=returnall)
  File "/home/jhunk/anaconda3/lib/python3.5/site-packages/conda/resolve.py", line 850, in solve
    reduced_index = self.get_reduced_index(specs)
  File "/home/jhunk/anaconda3/lib/python3.5/site-packages/conda/resolve.py", line 376, in get_reduced_index
    specs, features = self.verify_specs(specs)
  File "/home/jhunk/anaconda3/lib/python3.5/site-packages/conda/resolve.py", line 302, in verify_specs
    raise NoPackagesFoundError(bad_deps)
conda.exceptions.NoPackagesFoundError: Package missing in current osx-64 channels:
  - python x.x.*

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/Users/jhunk/anaconda3/bin/conda-build", line 6, in <module>
    sys.exit(conda_build.cli.main_build.main())
  File "/home/jhunk/anaconda3/lib/python3.5/site-packages/conda_build/cli/main_build.py", line 342, in main
    execute(sys.argv[1:])
  File "/home/jhunk/anaconda3/lib/python3.5/site-packages/conda_build/cli/main_build.py", line 333, in execute
    noverify=args.no_verify)
  File "/home/jhunk/anaconda3/lib/python3.5/site-packages/conda_build/api.py", line 183, in build
    need_source_download=need_source_download, config=config, variants=variants)
  File "/home/jhunk/anaconda3/lib/python3.5/site-packages/conda_build/build.py", line 1615, in build_tree
    built_packages=built_packages,
  File "/home/jhunk/anaconda3/lib/python3.5/site-packages/conda_build/build.py", line 928, in build
    channel_urls=tuple(m.config.channel_urls))
  File "/home/jhunk/anaconda3/lib/python3.5/site-packages/conda_build/environ.py", line 702, in get_install_actions
    raise DependencyNeedsBuildingError(exc, subdir=subdir)
conda_build.exceptions.DependencyNeedsBuildingError: Unsatisfiable dependencies for platform osx-64: ['python']
```




Discussion:
I think you're confusing numpy x.x with python x.x.  python x.x has never been supported.  There are issues with numpy x.x, and I'm working on those now.
Ok crap -- I guess I learn something new every day with this thing. We've been doing it wrong for years then.

In older versions if/when you specified `- python x.x` in the recipe but didn't issue `--python=[some_version]` then `conda build` would halt with a message like: "python x.x missing argument" or something to that effect.

Logic: At some point in 2015 `--python=` didn't work correctly so we used the `x.x` selector method as "insurance" to force conda build to grab the correct version of python at build-time.


I'm going to have to go back and search the source to understand why python x.x used to work.  The original "x.x" PR was https://github.com/conda/conda-build/pull/573

This comment especially is interesting: https://github.com/conda/conda-build/pull/573/files#r39464074

I think variants in conda-build 3 will make your life much better: https://conda.io/docs/building/variants.html
This does work on the 2.1.x, so I guess it was an undocumented feature.  I think it's not much effort to fix this.  I'll try to go ahead and make it work again, but it'll show a deprecation warning.
Hmm... so sort of half-fixed it.  It no longer throws any error, and it respects any --python argument you pass.  The issue is that it never raises an error - even if --python is missing.  How much of a problem will that be for you?  Basically, it doesn't work anymore because we're always setting it to the variant value, and there are always defaults for that, so it never raises an error.  There's no way to say "if x.x is present and python isn't set in the variant" because python is always set in the variant.

Resolving patch:
diff --git a/conda_build/environ.py b/conda_build/environ.py
--- a/conda_build/environ.py
+++ b/conda_build/environ.py
@@ -636,7 +636,7 @@ def package_specs(self):
         return specs
 
 
-spec_needing_star_re = re.compile("([0-9a-zA-Z\.]+\s+)([0-9a-zA-Z\.\+]+)(\s+[0-9a-zA-Z\.\_]+)?")
+spec_needing_star_re = re.compile("([0-9a-zA-Z\.]+)\s+([0-9a-zA-Z\.\+]+)(\s+[0-9a-zA-Z\.\_]+)?")
 spec_ver_needing_star_re = re.compile("^([0-9a-zA-Z\.]+)$")
 
 
@@ -644,17 +644,17 @@ def _ensure_valid_spec(spec):
     if isinstance(spec, MatchSpec):
         if (hasattr(spec, 'version') and spec.version and
                 spec_ver_needing_star_re.match(str(spec.version))):
-            if str(spec.name) != 'numpy' or str(spec.version) != 'x.x':
+            if str(spec.name) not in ('python', 'numpy') or str(spec.version) != 'x.x':
                 spec = MatchSpec("{} {}".format(str(spec.name), str(spec.version) + '.*'))
     else:
         match = spec_needing_star_re.match(spec)
         # ignore exact pins (would be a 3rd group)
         if match and not match.group(3):
-            if 'numpy' in match.group(1) and match.group(2) == 'x.x':
-                spec = spec_needing_star_re.sub(r"\1\2", spec)
+            if match.group(1) in ('python', 'numpy') and match.group(2) == 'x.x':
+                spec = spec_needing_star_re.sub(r"\1 \2", spec)
             else:
                 if "*" not in spec:
-                    spec = spec_needing_star_re.sub(r"\1\2.*", spec)
+                    spec = spec_needing_star_re.sub(r"\1 \2.*", spec)
     return spec
 
 
diff --git a/conda_build/render.py b/conda_build/render.py
--- a/conda_build/render.py
+++ b/conda_build/render.py
@@ -34,8 +34,6 @@
 from conda_build.index import get_build_index
 # from conda_build.jinja_context import pin_subpackage_against_outputs
 
-numpy_re = re.compile(r'^numpy\s*x\.x')
-
 
 def bldpkg_path(m):
     '''
@@ -64,7 +62,14 @@ def get_env_dependencies(m, env, variant, exclude_pattern=None,
         specs.extend([ms.spec for ms in m.ms_depends('host')])
     # replace x.x with our variant's numpy version, or else conda tries to literally go get x.x
     if env in ('build', 'host'):
-        specs = [spec.replace(' x.x', ' {}'.format(variant.get('numpy', ""))) for spec in specs]
+        no_xx_specs = []
+        for spec in specs:
+            if ' x.x' in spec:
+                pkg_name = spec.split()[0]
+                no_xx_specs.append(' '.join((pkg_name, variant.get(pkg_name, ""))))
+            else:
+                no_xx_specs.append(spec)
+        specs = no_xx_specs
     subpackages = []
     dependencies = []
     pass_through_deps = []
@@ -382,12 +387,14 @@ def insert_variant_versions(metadata, env):
                     del reqs[i]
                     reqs.insert(i, ' '.join((key, val)))
 
-    matches = [numpy_re.match(pkg) for pkg in reqs]
+    xx_re = re.compile("([0-9a-zA-Z\.\-\_]+)\s+x\.x")
+
+    matches = [xx_re.match(pkg) for pkg in reqs]
     if any(matches):
         for i, x in enumerate(matches):
             if x:
                 del reqs[i]
-                reqs.insert(i, ' '.join(('numpy', metadata.config.variant.get('numpy'))))
+                reqs.insert(i, ' '.join((x.group(1), metadata.config.variant.get(x.group(1)))))
     metadata.meta['requirements'][env] = reqs
 
 


Test patch:
diff --git a/tests/test-recipes/metadata/_python_xx/meta.yaml b/tests/test-recipes/metadata/_python_xx/meta.yaml
new file mode 100644
--- /dev/null
+++ b/tests/test-recipes/metadata/_python_xx/meta.yaml
@@ -0,0 +1,10 @@
+package:
+  name: conda-build-test-python-xx
+  version: 1.0
+
+requirements:
+  build:
+    # legacy "pin run as build"
+    - python x.x
+  run:
+    - python
diff --git a/tests/test-recipes/metadata/_python_xx/run_test.bat b/tests/test-recipes/metadata/_python_xx/run_test.bat
new file mode 100644
--- /dev/null
+++ b/tests/test-recipes/metadata/_python_xx/run_test.bat
@@ -0,0 +1,5 @@
+@echo on
+conda list -p "%PREFIX%" --canonical
+if errorlevel 1 exit 1
+conda list -p "%PREFIX%" --canonical | grep "conda-build-test-python-xx-1\.0-py34h......._0"
+if errorlevel 1 exit 1
diff --git a/tests/test-recipes/metadata/_python_xx/run_test.py b/tests/test-recipes/metadata/_python_xx/run_test.py
new file mode 100644
--- /dev/null
+++ b/tests/test-recipes/metadata/_python_xx/run_test.py
@@ -0,0 +1,24 @@
+import os
+import json
+import glob
+import sys
+
+
+def main():
+    prefix = os.environ['PREFIX']
+
+    info_files = glob.glob(os.path.join(prefix, 'conda-meta',
+                             'conda-build-test-python-xx-1.0-py34*0.json'))
+    assert len(info_files) == 1
+    info_file = info_files[0]
+    with open(info_file, 'r') as fh:
+        info = json.load(fh)
+
+    # numpy with no version, python with no version, python with version pin
+    depends = sorted(info['depends'])
+    assert depends[0].startswith('python ')
+    assert sys.version[:3] == ("3.4")
+
+
+if __name__ == '__main__':
+    main()
diff --git a/tests/test-recipes/metadata/_python_xx/run_test.sh b/tests/test-recipes/metadata/_python_xx/run_test.sh
new file mode 100644
--- /dev/null
+++ b/tests/test-recipes/metadata/_python_xx/run_test.sh
@@ -0,0 +1,2 @@
+conda list -p $PREFIX --canonical
+conda list -p $PREFIX --canonical | grep "conda-build-test-python-xx-1\.0-py34h......._0"
diff --git a/tests/test_api_build.py b/tests/test_api_build.py
--- a/tests/test_api_build.py
+++ b/tests/test_api_build.py
@@ -1083,3 +1083,8 @@ def test_numpy_xx(testing_config):
 def test_numpy_xx_host(testing_config):
     recipe = os.path.join(metadata_dir, '_numpy_xx_host')
     api.build(recipe, config=testing_config, numpy='1.12')
+
+
+def test_python_xx(testing_config):
+    recipe = os.path.join(metadata_dir, '_python_xx')
+    api.build(recipe, config=testing_config, python='3.4')