internlm/SWE-Fixer-Train-110K huggingface dataset:
Repository: conda/conda-build

Issue:
numpy matrix rendering in notebook example is much slower
Needs some profiling love


Discussion:

Resolving patch:
diff --git a/conda_build/utils.py b/conda_build/utils.py
--- a/conda_build/utils.py
+++ b/conda_build/utils.py
@@ -31,6 +31,7 @@
 from .conda_interface import PY3, iteritems
 from .conda_interface import root_dir, pkgs_dirs
 from .conda_interface import string_types, url_path, get_rc_urls
+from .conda_interface import memoized
 from .conda_interface import StringIO
 from .conda_interface import VersionOrder
 # NOQA because it is not used in this file.
@@ -701,6 +702,7 @@ def print_skip_message(metadata):
           "configuration.".format(metadata.path))
 
 
+@memoized
 def package_has_file(package_path, file_path):
     try:
         locks = get_conda_operation_locks()


Test patch:
diff --git a/tests/test_cli.py b/tests/test_cli.py
--- a/tests/test_cli.py
+++ b/tests/test_cli.py
@@ -106,7 +106,7 @@ def test_build_output_build_path(testing_workdir, testing_metadata, testing_conf
                                       sys.version_info.major, sys.version_info.minor, _hash))
     output, error = capfd.readouterr()
     # assert error == ""
-    assert output.rstrip() == test_path, error
+    assert test_path in output.rstrip(), error
 
 
 def test_build_output_build_path_multiple_recipes(testing_workdir, testing_metadata,
diff --git a/tests/test_variants.py b/tests/test_variants.py
--- a/tests/test_variants.py
+++ b/tests/test_variants.py
@@ -89,6 +89,15 @@ def test_variant_with_numpy_not_pinned_reduces_matrix():
     assert len(metadata) == 2
 
 
+def test_variant_with_numpy_pinned_has_matrix():
+    # variants are defined in yaml file in this folder
+    # there are two python versions and two numpy versions.  However, because numpy is not pinned,
+    #    the numpy dimensions should get collapsed.
+    recipe = os.path.join(recipe_dir, '04_numpy_matrix_pinned')
+    metadata = api.render(recipe)
+    assert len(metadata) == 4
+
+
 def test_pinning_in_build_requirements():
     recipe = os.path.join(recipe_dir, '05_compatible')
     metadata = api.render(recipe)[0][0]
Repository: conda/conda-build

Issue:
clean up git unable to describe output
When a git repo is not tagged, or has only lightweight tags, users see output like:

```
WARNING conda_build.environ:get_git_info(181): Command '['git', 'describe', '--tags', '--long', 'HEAD']' returned non-zero exit status
fatal: No names found, cannot describe anything. 
```

these are not generally errors, just warnings, but people have been concerned about it.  This error output should be hidden, if possible.


Discussion:

Resolving patch:
diff --git a/conda_build/build.py b/conda_build/build.py
--- a/conda_build/build.py
+++ b/conda_build/build.py
@@ -1068,7 +1068,7 @@ def build(m, post=None, need_source_download=True, need_reparse_in_env=False, bu
                                         subdir=subdir, is_cross=m.is_cross)
                         sub_build_ms_deps = m.ms_depends('build')
                     else:
-                        # When not cross-compiling, the build deps are the aggregate of 'build' and 'host'.
+                        # When not cross-compiling, the build deps aggregate 'build' and 'host'.
                         sub_build_ms_deps = m.ms_depends('build') + m.ms_depends('host')
                     index, index_timestamp = get_build_index(m.config, m.config.build_subdir)
                     build_actions = environ.get_install_actions(m.config.build_prefix, index,
diff --git a/conda_build/environ.py b/conda_build/environ.py
--- a/conda_build/environ.py
+++ b/conda_build/environ.py
@@ -72,7 +72,9 @@ def get_lua_include_dir(config):
 def verify_git_repo(git_dir, git_url, config, expected_rev='HEAD'):
     env = os.environ.copy()
     log = utils.get_logger(__name__)
-    if config.verbose:
+    base_log_level = log.getEffectiveLevel()
+
+    if config.debug:
         stderr = None
     else:
         FNULL = open(os.devnull, 'w')
@@ -147,6 +149,7 @@ def verify_git_repo(git_dir, git_url, config, expected_rev='HEAD'):
         log.warn(str(error))
         OK = False
     finally:
+        log.setLevel(base_log_level)
         if not config.verbose:
             FNULL.close()
     return OK
@@ -166,8 +169,9 @@ def get_git_info(repo, config):
     """
     d = {}
     log = utils.get_logger(__name__)
+    base_log_level = log.getEffectiveLevel()
 
-    if config.verbose:
+    if config.debug:
         stderr = None
     else:
         FNULL = open(os.devnull, 'w')
@@ -207,6 +211,7 @@ def get_git_info(repo, config):
         d['GIT_BUILD_STR'] = '{}_{}'.format(d["GIT_DESCRIBE_NUMBER"],
                                             d["GIT_DESCRIBE_HASH"])
 
+    log.setLevel(base_log_level)
     # issues on Windows with the next line of the command prompt being recorded here.
     assert not any("\n" in value for value in d.values())
     return d


Test patch:
Repository: conda/conda-build

Issue:
TypeError: <function configuration at 0x102a6a378> is not JSON serializable
A [recipe](https://github.com/biolab/orange3/blob/master/conda-recipe/meta.yaml) that builds fine with conda-build=2.1.15 throws an error when built with conda-build=2.1.16.

In the recipe, load_setup_py_data is used to read version information from a [setup.py](https://github.com/biolab/orange3/blob/master/setup.py) script. In the setup.py script, one of the [parameters to the setup function](https://github.com/biolab/orange3/blob/master/setup.py#L248) is a configuration function (used by numpy distutils).

As load_setup_py_data collects all parameters and tries to save them to a json file, it fails with a TypeError: <function configuration at 0x102a6a378> is not JSON serializable




Discussion:
Thanks for reporting. I know what caused this and will fix ASAP.
I am getting a similar error when setup.py defines an extension module:
```
TypeError: Object of type 'Extension' is not JSON serializable
```
This broke in commit f96aa7d8442bf54f33542b30a4f569de3e583202.

Resolving patch:
diff --git a/conda_build/_load_setup_py_data.py b/conda_build/_load_setup_py_data.py
--- a/conda_build/_load_setup_py_data.py
+++ b/conda_build/_load_setup_py_data.py
@@ -100,7 +100,6 @@ def setup(**kw):
     # we get back a dict of the setup data
     data = load_setup_py_data(**args.__dict__)
     with open(os.path.join(args.work_dir, 'conda_build_loaded_setup_py.json'), 'w') as f:
-        # need to remove the cmdclass as it is not json serializable
-        if 'cmdclass' in data:
-            del data['cmdclass']
-        json.dump(data, f)
+        # this is lossy.  Anything that can't be serialized is either forced to None or
+        #     removed completely.
+        json.dump(data, f, skipkeys=True, default=lambda x: None)


Test patch:
Repository: conda/conda-build

Issue:
CONDA_PY=27, 34 not recognized by conda-build 3.0
Those values now leads to a search for Python 27.* which fails as illustrated here https://travis-ci.org/Ecpy/conda-recipes/builds/251547219.
This is likely o break a number of continuous integration scripts and break backwards compatibility. Furthermore setting CONDA_PY=2.7 appears to work even if conda re-export CONDA_PY=27 as illustrated here https://travis-ci.org/Ecpy/conda-recipes/jobs/251550112.


Discussion:

Resolving patch:
diff --git a/conda_build/config.py b/conda_build/config.py
--- a/conda_build/config.py
+++ b/conda_build/config.py
@@ -160,6 +160,8 @@ def env(lang, default):
         def set_lang(variant, lang):
             value = env(lang, self.variant.get(lang))
             if value:
+                if '.' not in str(value):
+                    value = '.'.join((value[0], value[1:]))
                 variant[lang] = value
 
         # this is where we override any variant config files with the legacy CONDA_* vars


Test patch:
diff --git a/tests/test_cli.py b/tests/test_cli.py
--- a/tests/test_cli.py
+++ b/tests/test_cli.py
@@ -479,3 +479,10 @@ def test_no_force_upload(mocker, testing_workdir, testing_metadata):
         f.write('anaconda_upload: True\n')
     main_build.execute(args)
     assert call.called_once_with(['anaconda', 'upload', '--force', pkg])
+
+
+def test_conda_py_no_period(testing_workdir, testing_metadata, monkeypatch):
+    monkeypatch.setenv('CONDA_PY', '34')
+    api.output_yaml(testing_metadata, 'meta.yaml')
+    outputs = api.build(testing_workdir)
+    assert any('py34' in output for output in outputs)
Repository: conda/conda-build

Issue:
Build fails if both meta.yaml build script and build.sh are specified
This was introduced in conda-build 3.0. If you specify a build script and build.sh exists the build fails. Steps to reproduce:
* Add the following to a meta.yaml file:
```
build:
  script: python setup.py install
```
* `echo "python setup.py install" > build.sh`

Results in (just the useful part of the error):
```
running install_scripts
writing list of installed files to 'record.txt#!/bin/bash'
error: [Errno 2] No such file or directory: 'record.txt#!/bin/bash'
```
If you rename build.sh or remove the script line from meta.yaml the build works fine. This was not the behavior in 2.x. Seems like the current way (you can have one or the other but not both) is probably "right", but maybe the default behavior could be something better than breaking (took me a _while_ to debug this).



Discussion:
Thanks for reporting this. I'll think about how we want conda-build to decide between build.sh and the build script in meta.yaml, but I do think a warning of some sort is in order.

Resolving patch:
diff --git a/conda_build/build.py b/conda_build/build.py
--- a/conda_build/build.py
+++ b/conda_build/build.py
@@ -53,7 +53,7 @@
                               fix_permissions, get_build_metadata)
 
 from conda_build.index import update_index
-from conda_build.exceptions import indent, DependencyNeedsBuildingError
+from conda_build.exceptions import indent, DependencyNeedsBuildingError, CondaBuildException
 from conda_build.variants import (set_language_env_vars, dict_of_lists_to_list_of_dicts,
                                   get_package_variants)
 from conda_build.create_test import (create_files, create_shell_files, create_r_files,
@@ -1042,6 +1042,10 @@ def build(m, post=None, need_source_download=True, need_reparse_in_env=False, bu
                 windows.build(m, build_file)
             else:
                 build_file = join(m.path, 'build.sh')
+                if isfile(build_file) and script:
+                    raise CondaBuildException("Found a build.sh script and a build/script section"
+                                              "inside meta.yaml. Either remove the build.sh "
+                                              "script or remove the build/script section in meta.yaml.")
                 # There is no sense in trying to run an empty build script.
                 if isfile(build_file) or script:
 
@@ -1075,7 +1079,7 @@ def build(m, post=None, need_source_download=True, need_reparse_in_env=False, bu
                                                  m.config.host_prefix))
                         if script:
                                 bf.write(script)
-                        if isfile(build_file):
+                        if isfile(build_file) and not script:
                             bf.write(open(build_file).read())
 
                     os.chmod(work_file, 0o766)


Test patch:
Repository: conda/conda-build

Issue:
Set target in package section [feature request]
## Problem

Currently the only way to change the target platform is using the built in way of handling variants. This involves including an additional file `conda_build_config.yaml`. When you try to set `target` in the package section it is completely ignored and when you try to set `target` in the outputs section it throws an error.  Something like:
```
Expecting win-64 got linux-64
```

## Request

Ideally we would like to be able to set target directly in the package section, but our needs could be met as long as that functionality is exposed and not bound to the use of `conda_build_config.yaml`. I took a look at doing this myself but I am unfamiliar with the code base and the reliance on `target_platform` seems to be entrenched in the variants logic and spread across quite a few files.

Please let me know what you think!


Discussion:
Yes, I agree with you.  This needs to be fixed.  I think it'll need to be fixed by tweaking metadata.py's get_output_metadata method, so that the target attribute ends up overriding or setting the target_platform of the variant.  As you say, it is pretty tangled.  If a better overall structure for conda-build is obvious to you, please suggest it.  It has undergone a lot of organic growth and definitely could use a refactor.  Unfortunately, that's not a current priority.
Hey, I think I've found a solution for this but I can't find the docs on actually building the project to be able to test it. Basically because it's failing in `tarcheck.py` at an assertion statement simply adding `self.config.target_subdir` to the array in this line https://github.com/conda/conda-build/blob/8a7c41dd37204f0d6b3d31240f5180b617192b2f/conda_build/tarcheck.py#L85
to make it
```
assert info['subdir'] in [self.config.host_subdir, self.config.target_subdir, 'noarch'], \ 
```
should fix it.

I wanted to test for side effects, but all it's essentially doing is not failing the build if info.json's subdir == target_subdir which I believe is the behavior that is intended with this anyways

Of course that is just a short term fix to ensure that setting target in outputs actually works, I'll think more on the bigger picture refactoring.
Yes, I think you're right.  That fix is appropriate.

Resolving patch:
diff --git a/conda_build/tarcheck.py b/conda_build/tarcheck.py
--- a/conda_build/tarcheck.py
+++ b/conda_build/tarcheck.py
@@ -82,7 +82,7 @@ def prefix_length(self):
 
     def correct_subdir(self):
         info = json.loads(self.t.extractfile('info/index.json').read().decode('utf-8'))
-        assert info['subdir'] in [self.config.host_subdir, 'noarch'], \
+        assert info['subdir'] in [self.config.host_subdir, 'noarch', self.config.target_subdir], \
             ("Inconsistent subdir in package - index.json expecting {0},"
              " got {1}".format(self.config.host_subdir, info['subdir']))
 


Test patch:
Repository: conda/conda-build

Issue:
Files in binary_relocation list are not made relocatable
If the [binary_relocation](https://conda.io/docs/building/meta-yaml.html#binary-relocation) entry in a recipe is specified as a list of files, the files in the list are not made relocatable.

The logic responsible for this is found in [`post.py`](https://github.com/conda/conda-build/blob/9f5ee247b7aa8bccc9a4f667880a964ceb072c80/conda_build/post.py#L453-L457):

``` Python
    for f in files:
        if f.startswith('bin/'):
            fix_shebang(f, prefix=prefix, build_python=build_python, osx_is_app=osx_is_app)
        if binary_relocation is True or (isinstance(f, list) and f in binary_relocation):
            mk_relative(m, f, prefix)
```

The last if statement in this section should be modified to check if binary_relocation is a list not f:

``` Python
    for f in files:
        if f.startswith('bin/'):
            fix_shebang(f, prefix=prefix, build_python=build_python, osx_is_app=osx_is_app)
        if binary_relocation is True or (isinstance(binary_relocation, list) and f in binary_relocation):
            mk_relative(m, f, prefix)
```


Discussion:

Resolving patch:
diff --git a/conda_build/post.py b/conda_build/post.py
--- a/conda_build/post.py
+++ b/conda_build/post.py
@@ -453,7 +453,7 @@ def post_build(m, files, prefix, build_python, croot):
     for f in files:
         if f.startswith('bin/'):
             fix_shebang(f, prefix=prefix, build_python=build_python, osx_is_app=osx_is_app)
-        if binary_relocation is True or (isinstance(f, list) and f in binary_relocation):
+        if binary_relocation is True or (isinstance(binary_relocation, list) and f in binary_relocation):
             mk_relative(m, f, prefix)
 
 


Test patch:
Repository: conda/conda-build

Issue:
CONDA_PY variable format changed
...and broke my build :(

Previously, [as documented](https://conda.io/docs/user-guide/tasks/build-packages/environment-variables.html#environment-variables-that-affect-the-build-process) `CONDA_PY` was set to one of `27`, `34`, `35` or `36`.

It appears that in `conda-build=3.0.10=py36h21e6a50_0` a period has appeared in the middle of the number - e.g.
```
C:\Miniconda3\conda-bld\stanwell-utils_1503381692936\work>set "CONDA_PY=3.6"

C:\Miniconda3\conda-bld\stanwell-utils_1503381692936\work>set "PY3K=True"

C:\Miniconda3\conda-bld\stanwell-utils_1503381692936\work>set "PY_VER=3.6"
```

Was this an unintentional change? Can it be reverted? With the period in the middle it's no longer consistent with the package filename.

NB: Also seems to affect `PY_VER`


Discussion:
Sorry, that's a bug. Those variables will stay the same. I'll fix for next release.

Resolving patch:
diff --git a/conda_build/variants.py b/conda_build/variants.py
--- a/conda_build/variants.py
+++ b/conda_build/variants.py
@@ -139,12 +139,19 @@ def combine_variants(*variants):
 
 
 def set_language_env_vars(variant):
-    """Given args passed into conda command, set language env vars to be made available"""
+    """Given args passed into conda command, set language env vars to be made available.
+
+    Search terms: CONDA_PY, CONDA_R, CONDA_PERL, CONDA_LUA, CONDA_NPY
+    """
     inverse_map = {v: k for k, v in SUFFIX_MAP.items()}
     env = {}
     for variant_name, env_var_name in inverse_map.items():
         if variant_name in variant:
-            env['CONDA_' + env_var_name] = str(variant[variant_name])
+            value = str(variant[variant_name])
+            # legacy compatibility: python should be just first
+            if env_var_name == 'PY':
+                value = ''.join(value.split('.')[:2])
+            env['CONDA_' + env_var_name] = value
     return env
 
 


Test patch:
diff --git a/tests/test-recipes/metadata/python_env_defined/run_test.py b/tests/test-recipes/metadata/python_env_defined/run_test.py
--- a/tests/test-recipes/metadata/python_env_defined/run_test.py
+++ b/tests/test-recipes/metadata/python_env_defined/run_test.py
@@ -3,6 +3,7 @@
 assert os.getenv('PYTHON')
 assert os.getenv('PY_VER')
 assert os.getenv('CONDA_PY')
+assert '.' not in os.getenv('CONDA_PY')
 assert os.getenv('STDLIB_DIR')
 assert os.getenv('SP_DIR')
 assert os.getenv('PY3K')
Repository: conda/conda-build

Issue:
Dirty repodata.json after failed builds
When a package builds successfully but fails the test (overall fails the build) the package is removed from `conda-bld/linux-64/` however repodata.json still lists the package.

cc @msarahan 


Discussion:

Resolving patch:
diff --git a/conda_build/build.py b/conda_build/build.py
--- a/conda_build/build.py
+++ b/conda_build/build.py
@@ -45,7 +45,7 @@
 
 from conda_build import __version__
 from conda_build import environ, source, tarcheck, utils
-from conda_build.index import get_build_index
+from conda_build.index import get_build_index, update_index
 from conda_build.render import (output_yaml, bldpkg_path, render_recipe, reparse, finalize_metadata,
                                 distribute_variants, expand_outputs, try_download)
 import conda_build.os_utils.external as external
@@ -53,7 +53,6 @@
 from conda_build.post import (post_process, post_build,
                               fix_permissions, get_build_metadata)
 
-from conda_build.index import update_index
 from conda_build.exceptions import indent, DependencyNeedsBuildingError, CondaBuildException
 from conda_build.variants import (set_language_env_vars, dict_of_lists_to_list_of_dicts,
                                   get_package_variants)
@@ -1353,6 +1352,10 @@ def _construct_metadata_for_test_from_package(package, config):
     if not os.path.abspath(local_location):
         local_location = os.path.normpath(os.path.abspath(
             os.path.join(os.getcwd(), local_location)))
+
+    log = utils.get_logger(__name__)
+    log.info("Updating index at %s to make package installable with dependencies" % local_location)
+    update_index(local_location)
     local_url = url_path(local_location)
     # channel_urls is an iterable, but we don't know if it's a tuple or list.  Don't know
     #    how to add elements.
@@ -1633,7 +1636,11 @@ def tests_failed(package_or_metadata, move_broken, broken_dir, config):
     dest = join(broken_dir, os.path.basename(pkg))
 
     if move_broken:
+        log = utils.get_logger(__name__)
+        log.warn('Tests failed for %s - moving package to %s' % (os.path.basename(pkg),
+                 broken_dir))
         shutil.move(pkg, dest)
+        update_index(os.path.dirname(pkg))
     sys.exit("TESTS FAILED: " + os.path.basename(pkg))
 
 


Test patch:
Repository: conda/conda-build

Issue:
conda build --output doesn't respect --output-folder
I'm using conda 4.3.24 on Python 3.5.

When I run `conda build mypackage --output-folder pkgs/ --output`, the path conda returns is the filename the package would have if I hadn't specified the `--output-folder`, i.e., it points to the default conda-build `root-dir` as set in my `~/.condarc`.

This has the effect that I cannot use the path returned by the conda-build command to do further things with the package file.


Discussion:
Use --croot instead of --output-folder.  This is still a bug that I'll try to fix soon, but using --croot should get you further.
thanks, `conda build mypackage --output --croot pkgs/` does what I want!
Thanks for closing, but I'm going to reopen this until I have managed to make --output-folder behave as expected.
ah, sorry, I read your comment as "that's already recorded somewhere" -- my bad.

Resolving patch:
diff --git a/conda_build/render.py b/conda_build/render.py
--- a/conda_build/render.py
+++ b/conda_build/render.py
@@ -39,8 +39,8 @@ def bldpkg_path(m):
     '''
     Returns path to built package's tarball given its ``Metadata``.
     '''
-    output_dir = 'noarch' if m.noarch or m.noarch_python else m.config.host_subdir
-    return os.path.join(os.path.dirname(m.config.bldpkgs_dir), output_dir, '%s.tar.bz2' % m.dist())
+    subdir = 'noarch' if m.noarch or m.noarch_python else m.config.host_subdir
+    return os.path.join(m.config.output_folder, subdir, '%s.tar.bz2' % m.dist())
 
 
 def actions_to_pins(actions):


Test patch:
Repository: conda/conda-build

Issue:
source hash not verified
The md5 or sha256 hash of a source tarball is not checked prior to building a package.  For example the following recipe build successfully regardless of which md5 line is un-commented:

``` yaml
package:
  name: imagesize
  version: 0.7.1

source:
  fn: imagesize-0.7.1.tar.gz
  url: https://pypi.io/packages/source/i/imagesize/imagesize-0.7.1.tar.gz
  #md5: 976148283286a6ba5f69b0f81aef8052
  md5: deadbeefdeadbeefdeadbeefdeadbeef

build:
  number: 0
  script: python setup.py install --single-version-externally-managed --record record.txt

requirements:
  build:
    - python
    - setuptools
  run:
    - python
```

A build completes without an indication about an incorrect md5 with conda build `3.0.17`, it fails with conda build 2.1.17 with a notice about an MD5 mismatch.  

Will try other releases in the 3.x line shortly.


Discussion:
This behavior seems to have been introduced in the `3.0.11`, `3.0.12` or `3.0.13` release.  `3.0.10` gives an MD5 mismatch error where as `3.0.13` completes the build without any indication of an MD5 mismatch.  I'm not able to easily install `3.0.11` or `3.0.12` so I did not test those releases.

Resolving patch:
diff --git a/conda_build/source.py b/conda_build/source.py
--- a/conda_build/source.py
+++ b/conda_build/source.py
@@ -90,7 +90,7 @@ def download_to_cache(cache_folder, recipe_path, source_dict):
 
     hashed = None
     for tp in ('md5', 'sha1', 'sha256'):
-        if 'tp' in source_dict:
+        if tp in source_dict:
             expected_hash = source_dict[tp]
             hashed = hashsum_file(path, tp)
             if expected_hash != hashed:


Test patch: