internlm/SWE-Fixer-Train-110K huggingface dataset:
Repository: conda/conda-build

Issue:
conda build 2.0.0 - filelock fail on missing pkgs directory
Working with [conda build](https://github.com/conda/conda-build/tree/242e5accccea9196c58f10cb4ea7de21286b6b7b/conda_build):

```
version_version = '2.0.0'
version_full = '242e5accccea9196c58f10cb4ea7de21286b6b7b'
```

And getting the following traceback:

```
Traceback (most recent call last):
  File "/var/tmp/itwl/conda/tmp_envs/bd0dda3688f45d0a2adf/bin/conda-build-all", line 11, in <module>
    load_entry_point('conda-build-all==1.0.0rc1', 'console_scripts', 'conda-build-all')()
  File "/var/tmp/itwl/conda/tmp_envs/bd0dda3688f45d0a2adf/lib/python3.5/site-packages/conda_build_all/cli.py", line 90, in main
    b.main()
  File "/var/tmp/itwl/conda/tmp_envs/bd0dda3688f45d0a2adf/lib/python3.5/site-packages/conda_build_all/builder.py", line 263, in main
    built_dist_location = self.build(meta, build_config)
  File "/var/tmp/itwl/conda/tmp_envs/bd0dda3688f45d0a2adf/lib/python3.5/site-packages/conda_build_all/builder.py", line 200, in build
    conda_build.api.build(meta.meta, config=config)
  File "/var/tmp/itwl/conda/tmp_envs/bd0dda3688f45d0a2adf/lib/python3.5/site-packages/conda_build/api.py", line 81, in build
    need_source_download=need_source_download, config=config)
  File "/var/tmp/itwl/conda/tmp_envs/bd0dda3688f45d0a2adf/lib/python3.5/site-packages/conda_build/build.py", line 942, in build_tree
    config=config)
  File "/var/tmp/itwl/conda/tmp_envs/bd0dda3688f45d0a2adf/lib/python3.5/site-packages/conda_build/build.py", line 546, in build
    create_env(config.build_prefix, specs, config=config)
  File "/var/tmp/itwl/conda/tmp_envs/bd0dda3688f45d0a2adf/lib/python3.5/site-packages/conda_build/build.py", line 431, in create_env
    lock.acquire(timeout=config.timeout)
  File "/var/tmp/itwl/conda/tmp_envs/bd0dda3688f45d0a2adf/lib/python3.5/site-packages/filelock.py", line 243, in acquire
    raise Timeout(self._lock_file)
filelock.Timeout: The file lock '/var/tmp/itwl/conda/tmp_envs/bd0dda3688f45d0a2adf/pkgs/.conda_lock' could not be acquired.
```

The fail appears to be related to the fact that within `create_env` it is attempting to acquire a lock on a file within  the [dirname = os.path.join(cc.root_dir, 'pkgs')](https://github.com/conda/conda-build/blob/242e5accccea9196c58f10cb4ea7de21286b6b7b/conda_build/build.py#L427) directory and that `pkgs` root sub-directory does **not** exist.

Would it be valid for `create_env` to create the `pkgs` sub-directory if it doesn't exist, before attempting to acquire the filelock? Or is this issue the symptom of some other underlying problem?

Many thanks!



Discussion:

Resolving patch:
diff --git a/conda_build/build.py b/conda_build/build.py
--- a/conda_build/build.py
+++ b/conda_build/build.py
@@ -425,6 +425,8 @@ def create_env(prefix, specs, config, clear_cache=True):
 
             cc.pkgs_dirs = cc.pkgs_dirs[:1]
             dirname = os.path.join(cc.root_dir, 'pkgs')
+            if not os.path.isdir(dirname):
+                os.makedirs(dirname)
             lock_file = os.path.join(dirname, ".conda_lock")
 
             lock = filelock.SoftFileLock(lock_file)


Test patch:
Repository: conda/conda-build

Issue:
conda-build suggest to install beta version
Hi,

should it be always the released version?

``` bash
WARNING: conda-build appears to be out of date. You have version 1.21.11 but the
latest version is 2.0.0beta4. Run
```



Discussion:
To make this suggestion, conda-build looks at your package index.  Since 2.0.0beta4 is not in the default channels, you must have a conda-build 2.0.0beta4 package locally - one that you either built, or one you perhaps downloaded from the canary channel.

As far as always preferring released versions, that message comes from https://github.com/conda/conda-build/blob/master/conda_build/build.py#L490-L495 which looks pretty naive.  I agree with your inclination that it should prefer released versions.  It should be using whatever logic conda uses to prioritize versions.  I will add this to the milestone for the next release.


Resolving patch:
diff --git a/conda_build/build.py b/conda_build/build.py
--- a/conda_build/build.py
+++ b/conda_build/build.py
@@ -38,6 +38,7 @@
 from .conda_interface import Resolve, MatchSpec, NoPackagesFound, Unsatisfiable
 from .conda_interface import TemporaryDirectory
 from .conda_interface import get_rc_urls, get_local_urls
+from .conda_interface import VersionOrder
 
 from conda_build import __version__
 from conda_build import environ, source, tarcheck
@@ -487,7 +488,10 @@ def warn_on_old_conda_build(index):
         return
     r = Resolve(index)
     try:
-        pkgs = sorted(r.get_pkgs(MatchSpec('conda-build')))
+        pkgs = sorted(r.get_pkgs(MatchSpec('conda-build')), key=VersionOrder)
+        # cuts out packages wth rc/alpha/beta.  VersionOrder described in conda/version.py
+        # this breaks up the version into pieces, and depends on version formats like x.y.z[alpha/beta]  # noqa
+        pkgs = [pkg for pkg in pkgs if len(VersionOrder(pkg).version[3]) == 1]
     except NoPackagesFound:
         print("WARNING: Could not find any versions of conda-build in the channels", file=sys.stderr)  # noqa
         return
diff --git a/conda_build/conda_interface.py b/conda_build/conda_interface.py
--- a/conda_build/conda_interface.py
+++ b/conda_build/conda_interface.py
@@ -23,6 +23,7 @@
 from conda.utils import human_bytes, hashsum_file, md5_file, memoized, unix_path_to_win, win_path_to_unix, url_path  # NOQA
 import conda.config as cc  # NOQA
 from conda.config import sys_rc_path  # NOQA
+from conda.version import VersionOrder  # NOQA
 
 if parse_version(conda.__version__) >= parse_version("4.2"):
     # conda 4.2.x


Test patch:
Repository: conda/conda-build

Issue:
conda-inspect failure in latest conda-build (2.0.1)
Using `conda-build 2.0.1` I get the following error. This does not happen every time so I guess it is related to how `conda-inspect` is handling errors like the `libuuid.so.1: no version information` below. 
(Note that the same command works for `conda-build 1.21.11`).

``` shell
conda-inspect linkages -n _test pyferret
Using Anaconda API: https://api.anaconda.org
/opt/conda/envs/_test/lib/python2.7/site-packages/pyferret/libpyferret.so: /opt/conda/envs/_test/lib/python2.7/site-packages/pyferret/../../.././libuuid.so.1: no version information available (required by /usr/lib64/libSM.so.6)
Traceback (most recent call last):
  File "/opt/conda/bin/conda-inspect", line 6, in <module>
    sys.exit(conda_build.cli.main_inspect.main())
  File "/opt/conda/lib/python3.5/site-packages/conda_build/cli/main_inspect.py", line 190, in main
    return execute(sys.argv[1:])
  File "/opt/conda/lib/python3.5/site-packages/conda_build/cli/main_inspect.py", line 179, in execute
    show_files=args.show_files, groupby=args.groupby))
  File "/opt/conda/lib/python3.5/site-packages/conda_build/api.py", line 223, in inspect_linkages
    show_files=show_files, groupby=groupby)
  File "/opt/conda/lib/python3.5/site-packages/conda_build/inspect.py", line 242, in inspect_linkages
    comma_join(deps), file=sys.stderr)
  File "/opt/conda/lib/python3.5/logging/__init__.py", line 1296, in warn
    self.warning(msg, *args, **kwargs)
  File "/opt/conda/lib/python3.5/logging/__init__.py", line 1291, in warning
    self._log(WARNING, msg, args, **kwargs)
TypeError: _log() got an unexpected keyword argument 'file'
```

Ping @msarahan 

PS: I am loving `conda-build 2`!



Discussion:

Resolving patch:
diff --git a/conda_build/inspect.py b/conda_build/inspect.py
--- a/conda_build/inspect.py
+++ b/conda_build/inspect.py
@@ -239,7 +239,7 @@ def inspect_linkages(packages, prefix=sys.prefix, untracked=False,
                     deps = list(which_package(path))
                     if len(deps) > 1:
                         log.warn("Warning: %s comes from multiple packages: %s", path,
-                                 comma_join(deps), file=sys.stderr)
+                                 comma_join(deps))
                     if not deps:
                         if exists(path):
                             depmap['untracked'].append((lib, path.split(prefix +


Test patch:
Repository: conda/conda-build

Issue:
Passing recipe dir with trailing slash leads to blank build id
basing build id on parent folder perhaps isn't the best way, anyway.



Discussion:

Resolving patch:
diff --git a/conda_build/build.py b/conda_build/build.py
--- a/conda_build/build.py
+++ b/conda_build/build.py
@@ -979,6 +979,7 @@ def build_tree(recipe_list, config, build_only=False, post=False, notest=False,
             to_build_recursive.append(metadata.name())
         else:
             recipe_parent_dir = os.path.dirname(recipe)
+            recipe = recipe.rstrip("/").rstrip("\\")
             recipe_config = config
             to_build_recursive.append(os.path.basename(recipe))
 
diff --git a/conda_build/config.py b/conda_build/config.py
--- a/conda_build/config.py
+++ b/conda_build/config.py
@@ -215,6 +215,7 @@ def compute_build_id(self, package_name, reset=False):
 
     @build_id.setter
     def build_id(self, _build_id):
+        _build_id = _build_id.rstrip("/").rstrip("\\")
         assert not os.path.isabs(_build_id), ("build_id should not be a absolute path, "
                                               "to preserve croot during path joins")
         self._build_id = _build_id


Test patch:
diff --git a/tests/test_cli.py b/tests/test_cli.py
--- a/tests/test_cli.py
+++ b/tests/test_cli.py
@@ -87,6 +87,19 @@ def test_build_output_build_path_multiple_recipes(testing_workdir, test_config,
     assert error == ""
     assert output.rstrip().splitlines() == test_paths, error
 
+def test_slash_in_recipe_arg_keeps_build_id(testing_workdir, test_config):
+    recipe_path = os.path.join(metadata_dir, "has_prefix_files" + os.path.sep)
+    fn = api.get_output_file_path(recipe_path, config=test_config)
+    args = [os.path.join(metadata_dir, "has_prefix_files"), '--croot', test_config.croot]
+    main_build.execute(args)
+    fn = api.get_output_file_path(recipe_path,
+                                  config=test_config)
+    assert package_has_file(fn, 'info/has_prefix')
+    data = package_has_file(fn, 'info/has_prefix')
+    if hasattr(data, 'decode'):
+        data = data.decode('UTF-8')
+    assert 'has_prefix_files_1' in data
+
 
 def test_render_output_build_path_set_python(testing_workdir, capfd):
     # build the other major thing, whatever it is
Repository: conda/conda-build

Issue:
LinkError from conda 4.2 not handled for prefix length retry
When OpenSSL's custom post-link script runs, if the build prefix is too long, the error that we end up seeing is:

File "/home/vagrant/miniconda2/lib/python2.7/site-packages/conda/install.py", line 990, in link
    raise LinkError("Error: post-link failed for: %s" % dist)
LinkError: Link error: Error: post-link failed for: openssl-1.0.1k-1

Conda-build needs to catch this error in https://github.com/conda/conda-build/blob/master/conda_build/build.py#L426

CC @kalefranz @soapy1 



Discussion:

Resolving patch:
diff --git a/conda_build/build.py b/conda_build/build.py
--- a/conda_build/build.py
+++ b/conda_build/build.py
@@ -38,7 +38,7 @@
 from .conda_interface import TemporaryDirectory
 from .conda_interface import get_rc_urls, get_local_urls
 from .conda_interface import VersionOrder
-from .conda_interface import PaddingError
+from .conda_interface import PaddingError, LinkError
 
 from conda_build import __version__
 from conda_build import environ, source, tarcheck
@@ -423,7 +423,7 @@ def create_env(prefix, specs, config, clear_cache=True):
                     for k, v in os.environ.items():
                         os.environ[k] = str(v)
                 plan.execute_actions(actions, index, verbose=config.debug)
-            except (SystemExit, PaddingError) as exc:
+            except (SystemExit, PaddingError, LinkError) as exc:
                 if (("too short in" in str(exc) or
                         'post-link failed for: openssl' in str(exc) or
                         isinstance(exc, PaddingError)) and
diff --git a/conda_build/conda_interface.py b/conda_build/conda_interface.py
--- a/conda_build/conda_interface.py
+++ b/conda_build/conda_interface.py
@@ -51,6 +51,7 @@
     get_local_urls = lambda: list(get_conda_build_local_url()) or []
     load_condarc = lambda fn: conda.base.context.reset_context([fn])
     PaddingError = conda.exceptions.PaddingError
+    LinkError = conda.exceptions.PaddingError
 
 else:
     from conda.config import get_default_urls, non_x86_linux_machines, load_condarc  # NOQA
@@ -74,6 +75,9 @@
     class PaddingError(Exception):
         pass
 
+    class LinkError(Exception):
+        pass
+
 
 class SignatureError(Exception):
     pass


Test patch:
Repository: conda/conda-build

Issue:
Jinja errors in package name evaluation are masked
Jinja errors inside package name expression are overridden / masked by `Error: package/name missing in ...`. See:

``` yaml
package:
  name: {{ some_erroneous_expression }}
```

``` bash
+(root) $ conda build --no-anaconda-upload . 1>/dev/null 
Error: package/name missing in: '/home/maba/dev/conda/conda-build/bugs/jinja-package-name-error-masked/meta.yaml'
```

Notice that errors for other fields are not masked:

``` yaml
package:
  name: some_package
  version: {{ some_erroneous_expression }}                                                                               
```

``` bash
+(root) $conda build --no-anaconda-upload . 1>/dev/null 
Undefined Jinja2 variables remain (['some_erroneous_expression']).  Please enable source downloading and try again.
```



Discussion:
Thanks for raising this.  The name field is special cased awkwardly.  I will look into addressing this, but I'm not sure how much trouble it will cause.  It's a matter of how many places the name is used before the jinja rendering is completed.  It would be nice to fix this, but it might be too much work right now, compared to the inconvenience of having to hardcode the name field.

My main concern is #1379 and https://github.com/conda/conda-build/issues/1379#issuecomment-248826521, that is from the user's perspective every thing should be fine but a `package/name missing` error gets put out. I agree it is not a high priority issue since hardcoding a single field should be quite tolerable! Though having a non-working example is of course a bit misleading, so at least the example should be adjusted in the meantime.


Resolving patch:
diff --git a/conda_build/environ.py b/conda_build/environ.py
--- a/conda_build/environ.py
+++ b/conda_build/environ.py
@@ -335,7 +335,8 @@ def meta_vars(meta, config):
     elif external.find_executable('hg', config.build_prefix) and os.path.exists(hg_dir):
         d.update(get_hg_build_info(hg_dir))
 
-    d['PKG_NAME'] = meta.name()
+    # use `get_value` to prevent early exit while name is still unresolved during rendering
+    d['PKG_NAME'] = meta.get_value('package/name')
     d['PKG_VERSION'] = meta.version()
     d['PKG_BUILDNUM'] = str(meta.build_number())
     d['PKG_BUILD_STRING'] = str(meta.build_id())


Test patch:
Repository: conda/conda-build

Issue:
--output reports wrong folder for noarch packages
In linux-64 packages build with the noarch  flag end up in the appropriate directory, but the --output function does not return the correct path.  



Discussion:
Thanks for the report. We'll fix this for the next bugfix release.

On Sep 19, 2016 1:24 PM, "Matty G" notifications@github.com wrote:

> In linux-64 packages build with the noarch flag end up in the appropriate
> directory, but the --output function does not return the correct path.
> 
> —
> You are receiving this because you are subscribed to this thread.
> Reply to this email directly, view it on GitHub
> https://github.com/conda/conda-build/issues/1373, or mute the thread
> https://github.com/notifications/unsubscribe-auth/AACV-aSffCKyIm7EvzdtG5_XJjO9vOjtks5qrtNIgaJpZM4KAzpW
> .


Resolving patch:
diff --git a/conda_build/render.py b/conda_build/render.py
--- a/conda_build/render.py
+++ b/conda_build/render.py
@@ -69,7 +69,8 @@ def bldpkg_path(m, config):
     '''
     Returns path to built package's tarball given its ``Metadata``.
     '''
-    return os.path.join(config.bldpkgs_dir, '%s.tar.bz2' % m.dist())
+    output_dir = m.info_index()['subdir']
+    return os.path.join(os.path.dirname(config.bldpkgs_dir), output_dir, '%s.tar.bz2' % m.dist())
 
 
 def parse_or_try_download(metadata, no_download_source, config,


Test patch:
diff --git a/tests/test_render.py b/tests/test_render.py
--- a/tests/test_render.py
+++ b/tests/test_render.py
@@ -1,9 +1,18 @@
-import os
-import subprocess
-import sys
+from conda_build import api
 
-thisdir = os.path.dirname(os.path.realpath(__file__))
-metadata_dir = os.path.join(thisdir, "test-recipes/metadata")
+from .utils import testing_workdir, test_config, test_metadata
+
+
+def test_output_with_noarch_says_noarch(test_metadata):
+    test_metadata.meta['noarch'] = 'python'
+    output = api.get_output_file_path(test_metadata)
+    assert 'noarch' in output
+
+
+def test_output_with_noarch_python_says_noarch(test_metadata):
+    test_metadata.meta['noarch_python'] = True
+    output = api.get_output_file_path(test_metadata)
+    assert 'noarch' in output
 
 
 # no tests here - this is tested at a high level in test_cli.py and in test_api_render.py.
Repository: conda/conda-build

Issue:
Getting a "Error: no such patch:" when the field is empty
@msarahan I noticed that latest `conda-build` is failing when a field is empty for certain platforms. This used to work:

```yaml
  patches:
    - skip_failing_test.patch  # [osx]
    - makefile.vc.patch  # [win]
```

but now I need to add a `# [not linux]` to the ` patches` filed otherwise  I get:

```
Source cache directory is: /opt/conda/conda-bld/src_cache
Downloading source to cache: libspatialite-4.3.0a.tar.gz
Downloading http://www.gaia-gis.it/gaia-sins/libspatialite-4.3.0a.tar.gz
Success
Error: no such patch: /conda-recipes/recipes/libspatialite/

./scripts/run_docker_build.sh returned exit code 1
```

I see similar failures for tests too.

recipe: https://github.com/ioos/conda-recipes/blob/master/recipes/libspatialite/meta.yaml#L11

CI log: https://circleci.com/gh/ioos/conda-recipes/1363


Discussion:

Resolving patch:
diff --git a/conda_build/utils.py b/conda_build/utils.py
--- a/conda_build/utils.py
+++ b/conda_build/utils.py
@@ -598,6 +598,9 @@ def package_has_file(package_path, file_path):
 
 def ensure_list(arg):
     from .conda_interface import string_types
-    if isinstance(arg, string_types) or not hasattr(arg, '__iter__'):
-        arg = [arg]
+    if (isinstance(arg, string_types) or not hasattr(arg, '__iter__')):
+        if arg:
+            arg = [arg]
+        else:
+            arg = []
     return arg


Test patch:
diff --git a/tests/test-recipes/metadata/empty_patch_section/meta.yaml b/tests/test-recipes/metadata/empty_patch_section/meta.yaml
new file mode 100644
--- /dev/null
+++ b/tests/test-recipes/metadata/empty_patch_section/meta.yaml
@@ -0,0 +1,9 @@
+package:
+  name: patch_section_empty
+  version: 1.0
+
+source:
+  path: .
+  # the test here is that selectors can make this field empty.  Make it empty here no matter what.
+  #    https://github.com/conda/conda-build/issues/1490
+  patches:
Repository: conda/conda-build

Issue:
`test source_files` does not create intermediate directories
I have the following part in my `meta.yml`

```yml
test:
  source_files:
    - ipynbtest/examples/ipynbtest_tutorial.ipynb
```

This fails with an error like: (I replaced my homefolder with `~` there are actually absolute paths)
```
IOError: [Errno 2] No such file or directory: u'~/anaconda/conda-bld/conda-recipe_1478272007306/test_tmp/ipynbtest/examples/ipynbtest_tutorial.ipynb'
```

After some digging it turns out that the problem is that in `test_tmp` there is no `examples` directory and hence the 

```
  File "~/anaconda/lib/python2.7/site-packages/conda_build/create_test.py", line 69, in create_files
    copy_into(f, f.replace(config.work_dir, config.test_dir), config.timeout)
```

can find the source but not open the destination

```
  File "~/anaconda/lib/python2.7/shutil.py", line 83, in copyfile with open(dst, 'wb') as fdst:
```

When change the part in `meta.yaml` to 

```yml
test:
  source_files:
    - ipynbtest/examples/*.ipynb
```

it also _fails_ whereas 

```yml
test:
  source_files:
    - ipynbtest/examples
```

it works. 

My conclusion is that this needs to be either well documented or better the intermediate folders needs to be created. This should be an easy fix if so desired.


Discussion:
I agree that the intermediate folders need to be created.  That's pretty easy and it makes user's lives much easier.

Feel free to submit a PR.  Otherwise, I'll try to get to it soon.

I can give it a try. Not so familiar with the complete code of conda build. If adding it to the `copy_into` function maybe with a option to turn this off (default would be on) I can see if that does it.

Sure, I don't really see the need to turn it off though - just precede any file copies (as opposed to directory copies/merges) with a

```
try:
    os.makedirs(os.path.dirname(file_to_copy))
```

of course, it's more annoying than that due to some files not having a parent folder (files at root level), but I'm sure you get the idea.


Resolving patch:
diff --git a/conda_build/utils.py b/conda_build/utils.py
--- a/conda_build/utils.py
+++ b/conda_build/utils.py
@@ -113,6 +113,14 @@ def copy_into(src, dst, timeout=90, symlinks=False, lock=None):
         if not lock:
             lock = get_lock(src_folder, timeout=timeout)
         with lock:
+            # if intermediate folders not not exist create them
+            dst_folder = os.path.dirname(dst)
+            if dst_folder and not os.path.exists(dst_folder):
+                try:
+                    os.makedirs(dst_folder)
+                except OSError:
+                    pass
+
             # with each of these, we are copying less metadata.  This seems to be necessary
             #   to cope with some shared filesystems with some virtual machine setups.
             #  See https://github.com/conda/conda-build/issues/1426


Test patch: