Repository: conda-forge/conda-smithy

Issue:
conda-smithy adds R 3.4.2 to build matrix
conda-forge has r-base 3.3.2 and 3.4.1 available:

https://anaconda.org/conda-forge/r-base/files

However, `conda smithy rerender` sometimes adds R 3.4.2 to the build matrix. This is presumably because R 3.4.2 is now available in the `r` channel:

https://anaconda.org/r/r-base/files

Here's an an example rerendering PR that adds R 3.4.2:

https://github.com/conda-forge/r-ape-feedstock/pull/1

A similar issue is happening on staged-recipes, where recipes for new R packages are being built with R 3.4.2:

https://github.com/conda-forge/staged-recipes/issues/4320

Is it possible to limit the build matrix to only include R versions available on the conda-forge channel?


Discussion:
@jakirkham @ocefpaf can we revert this? Or at least coordinate this better? I don't think we want to rebuild everything against R3.4.2. Thats a lot of work with little gain.
@jdblischak, @bgruening, the line that needs to be changed is https://github.com/conda-forge/conda-smithy/blob/master/conda_smithy/configure_feedstock.py#L640 to disallow `3.4.2`.
Thanks @isuruf! Is this the correct syntax to only allow 3.3.2 and 3.4.1: `'r-base =3.3.2|3.4.1'`
@jdblischak, I'm not sure. Can you try running `conda build-all recipe --matrix-conditions 'r-base =3.3.2|3.4.1'` in one of those feedstocks and check whether `3.4.2` is built or not?
@isuruf Testing it out now
@isuruf So the single `=` is invalid syntax, which is fixed by using `==`. However, even though conda-smithy adds R 3.4.2, I can't get `conda-build-all` to build anything other than R 3.3.2. Here's a summary of what I did:

```
git clone git@github.com:conda-forge/r-ape-feedstock.git
cd r-ape-feedstock
# Confirm that R 3.4.2 is added by conda-smithy
conda smithy rerender
git diff --staged
git reset --hard
# All of the following result in only R 3.3.2 being built
conda build-all recipe --matrix-conditions 'r-base ==3.3.2'
conda build-all recipe --matrix-conditions 'r-base ==3.4.1'
conda build-all recipe --matrix-conditions 'r-base ==3.3.2|3.4.1'
```

Should conda-build-all be able to run this successfully? After it displays the build, it always throws an error, even when I use the current `r-base >=3.3.2`:

```
$ conda build-all recipe --matrix-conditions 'r-base >=3.3.2'
Fetching package metadata .....
Resolving distributions from 1 recipes... 
Computed that there are 1 distributions from the 1 recipes:
Resolved dependencies, will be built in the following order: 
	r-ape-4.1-r3.3.2_0 (will be built: True)
Building  r-ape-4.1-r3.3.2_0
BUILD START: r-ape-4.1-r3.3.2_0
    (actual version deferred until further download or env creation)
Traceback (most recent call last):
  File "/home/jdb-work/miniconda3-conda-forge/lib/python3.6/site-packages/conda_build/build.py", line 1524, in build_tree
    config=config)
  File "/home/jdb-work/miniconda3-conda-forge/lib/python3.6/site-packages/conda_build/build.py", line 1030, in build
    create_env(config.build_prefix, specs, config=config)
  File "/home/jdb-work/miniconda3-conda-forge/lib/python3.6/site-packages/conda_build/build.py", line 675, in create_env
    actions = install_actions(prefix, index, specs)
  File "/home/jdb-work/miniconda3-conda-forge/lib/python3.6/site-packages/conda/plan.py", line 461, in install_actions
    update_deps, pinned)
  File "/home/jdb-work/miniconda3-conda-forge/lib/python3.6/site-packages/conda/plan.py", line 634, in get_actions_for_dists
    pkgs = r.install(specs, installed, update_deps=update_deps)
  File "/home/jdb-work/miniconda3-conda-forge/lib/python3.6/site-packages/conda/resolve.py", line 813, in install
    pkgs = self.solve(specs, returnall=returnall)
  File "/home/jdb-work/miniconda3-conda-forge/lib/python3.6/site-packages/conda/resolve.py", line 853, in solve
    reduced_index = self.get_reduced_index(specs)
  File "/home/jdb-work/miniconda3-conda-forge/lib/python3.6/site-packages/conda/resolve.py", line 374, in get_reduced_index
    specs, features = self.verify_specs(specs)
  File "/home/jdb-work/miniconda3-conda-forge/lib/python3.6/site-packages/conda/resolve.py", line 300, in verify_specs
    raise ResolvePackageNotFound(bad_deps)
conda.exceptions.ResolvePackageNotFound: 
  - r-base 3.3.2*
  - libgcc
  - r-lattice
  - r-base 3.4.1*
  - libgcc
  - r-nlme
  - r-base 3.4.1*
  - libgcc
  - r-nlme
  - r-lattice
  - gcc

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/jdb-work/miniconda3-conda-forge/bin/conda-build-all", line 6, in <module>
    exit(main())
  File "/home/jdb-work/miniconda3-conda-forge/lib/python3.6/site-packages/conda_build_all/cli.py", line 93, in main
    b.main()
  File "/home/jdb-work/miniconda3-conda-forge/lib/python3.6/site-packages/conda_build_all/builder.py", line 277, in main
    built_dist_location = self.build(meta, build_config)
  File "/home/jdb-work/miniconda3-conda-forge/lib/python3.6/site-packages/conda_build_all/builder.py", line 209, in build
    conda_build.api.build(meta.meta, config=config)
  File "/home/jdb-work/miniconda3-conda-forge/lib/python3.6/site-packages/conda_build/api.py", line 97, in build
    need_source_download=need_source_download, config=config)
  File "/home/jdb-work/miniconda3-conda-forge/lib/python3.6/site-packages/conda_build/build.py", line 1567, in build_tree
    .format(recipe, error_str) + "\n\n" + extra_help)
RuntimeError: Can't build {'undefined_jinja_vars': [], 'config': <conda_build.config.Config object at 0x7f2341e46518>, 'meta_path': '/home/jdb-work/Desktop/r-ape-feedstock/recipe/meta.yaml', 'path': '/home/jdb-work/Desktop/r-ape-feedstock/recipe', 'requirements_path': '/home/jdb-work/Desktop/r-ape-feedstock/recipe/requirements.txt', 'meta': {'package': {'name': 'r-ape', 'version': '4.1'}, 'source': {'fn': 'ape_4.1.tar.gz', 'url': ['https://cran.r-project.org/src/contrib/ape_4.1.tar.gz', 'https://cran.r-project.org/src/contrib/Archive/ape/ape_4.1.tar.gz'], 'sha256': '935af5ddadcba832d3f9cc032a80fc1a2e627a7ed54ef5f3773f87e06374a924'}, 'build': {'number': '0', 'rpaths': ['lib/R/lib/', 'lib/']}, 'requirements': {'build': ['r-base', 'r-lattice', 'r-nlme', 'gcc'], 'run': ['r-base', 'r-lattice', 'r-nlme', 'libgcc']}, 'test': {'commands': ['$R -e "library(\'ape\')"']}, 'about': {'home': 'http://ape-package.ird.fr/', 'license': 'GPL (>= 2)', 'summary': "Functions for reading, writing, plotting, and manipulating phylogenetic trees, analyses of comparative data in a phylogenetic framework, ancestral character analyses, analyses of diversification and macroevolution, computing distances from DNA sequences, reading and writing nucleotide sequences as well as importing from BioConductor, and several tools such as Mantel's test, generalized skyline plots, graphical exploration of phylogenetic data (alex, trex, kronoviz), estimation of absolute evolutionary rates and clock-like trees using mean path lengths and penalized likelihood, dating trees with non-contemporaneous sequences, translating DNA into AA sequences, and assessing sequence alignments. Phylogeny estimation can be done with the NJ, BIONJ, ME, MVR, SDM, and triangle methods, and several methods handling incomplete distance matrices (NJ*, BIONJ*, MVR*, and the corresponding triangle method). Some functions call external applications (PhyML, Clustal, T-Coffee, Muscle) whose results are returned into R.", 'license_family': 'GPL3'}, 'extra': {'recipe-maintainers': ['johanneskoester', 'bgruening'], 'final': True}}} due to unsatisfiable dependencies:

  - r-base 3.3.2*
  - libgcc
  - r-lattice
  - r-base 3.4.1*
  - libgcc
  - r-nlme
  - r-base 3.4.1*
  - libgcc
  - r-nlme
  - r-lattice
  - gcc


```

In case it could be useful, here are the package versions I am using:

```
$ conda list | grep -e ^conda 
conda                     4.3.29                   py36_0    conda-forge
conda-build               2.1.17                   py36_0    conda-forge
conda-build-all           1.1.1                      py_2    conda-forge
conda-env                 2.6.0                         0    conda-forge
conda-execute             0.8.1                    py36_2    conda-forge
conda-smithy              2.4.2                      py_0    conda-forge
conda-verify              2.0.0                    py36_0    conda-forge
```
Sorry it should have been `conda build-all . --matrix-conditions 'r-base >=3.3.2'`.
> Sorry it should have been `conda build-all r-ape-feedstock --matrix-conditions 'r-base >=3.3.2'`.

@isuruf I get the same behavior whether I provide the absolute path to the recipe subdirectory, the feedstock directory, or the upstream directory that contains the feedstock repo.
For me, `conda build-all . --matrix-conditions 'r-base >=3.3.2,<3.4.2'` and `conda build-all . --matrix-conditions 'r-base >=3.3.2,!=3.4.2'` both builds 3.3.2 and 3.4.1, but doesn't build 3.4.2.
`conda build-all . --matrix-conditions 'r-base >=3.3.2'` builds all 3 versions
@isuruf Thanks for your help! I realized the error I was getting was due to packages being installed from different channels in my `.condarc`. Creating a pure conda-forge setup fixed the problem.

I prefer `'r-base ==3.3.2|==3.4.1'` because it is more explicit and easier to maintain in the future. When a new version of r-base is released by conda forge, e.g. 3.5.1, this can be easily added to the build matrix with `'r-base ==3.3.2|==3.4.1|==3.5.1'`.

```
$ conda build-all . --matrix-conditions 'r-base ==3.3.2|==3.4.1' --dry-run
Fetching package metadata .................
Resolving distributions from 1 recipes... 
Computed that there are 2 distributions from the 1 recipes:
Resolved dependencies, will be built in the following order: 
	r-ape-4.1-r3.3.2_0 (will be built: False)
	r-ape-4.1-r3.4.1_0 (will be built: False)
Dry run: no distributions built
```
Can you send a PR?

Resolving patch:
diff --git a/conda_smithy/configure_feedstock.py b/conda_smithy/configure_feedstock.py
--- a/conda_smithy/configure_feedstock.py
+++ b/conda_smithy/configure_feedstock.py
@@ -637,7 +637,7 @@ def compute_build_matrix(meta, existing_matrix=None, channel_sources=tuple()):
     index = conda.api.get_index(channel_urls=channel_sources,
                                 platform=meta_config(meta).subdir)
     mtx = special_case_version_matrix(meta, index)
-    mtx = list(filter_cases(mtx, ['python >=2.7,<3|>=3.5', 'numpy >=1.11', 'r-base >=3.3.2']))
+    mtx = list(filter_cases(mtx, ['python >=2.7,<3|>=3.5', 'numpy >=1.11', 'r-base ==3.3.2|==3.4.1']))
     if existing_matrix:
         mtx = [tuple(mtx_case) + tuple(MatrixCaseEnvVar(*item) for item in case)
                for case in sorted(existing_matrix)


Test patch: