internlm/SWE-Fixer-Train-110K huggingface dataset:
Repository: conda/conda-build

Issue:
Conda skeleton occasionally chokes on cpan module
_From @jerowe on November 8, 2016 5:49_

```
conda skeleton cpan Devel::CheckBin
```

```
INFO:stdoutlog:

.INFO:requests.packages.urllib3.connectionpool:Starting new HTTP connection (1): api.metacpan.org
INFO:requests.packages.urllib3.connectionpool:Starting new HTTP connection (1): api.metacpan.org
INFO:requests.packages.urllib3.connectionpool:Starting new HTTP connection (1): api.metacpan.org
Traceback (most recent call last):
  File "/home/jillian/.local/easybuild/software/Anaconda3/4.0.0/bin/conda-skeleton", line 6, in <module>
    sys.exit(conda_build.cli.main_skeleton.main())
  File "/home/jillian/.local/easybuild/software/Anaconda3/4.0.0/lib/python3.5/site-packages/conda_build/cli/main_skeleton.py", line 65, in main
    return execute(sys.argv[1:])
  File "/home/jillian/.local/easybuild/software/Anaconda3/4.0.0/lib/python3.5/site-packages/conda_build/cli/main_skeleton.py", line 61, in execute
    api.skeletonize(package, args.repo, config=config)
  File "/home/jillian/.local/easybuild/software/Anaconda3/4.0.0/lib/python3.5/site-packages/conda_build/api.py", line 193, in skeletonize
    recursive=recursive, config=config, **kwargs)
  File "/home/jillian/.local/easybuild/software/Anaconda3/4.0.0/lib/python3.5/site-packages/conda_build/skeletons/cpan.py", line 301, in skeletonize
    recursive=recursive, config=config)
  File "/home/jillian/.local/easybuild/software/Anaconda3/4.0.0/lib/python3.5/site-packages/conda_build/skeletons/cpan.py", line 493, in deps_for_package
    dep_version, perl_version, dependency=True, config=config)
  File "/home/jillian/.local/easybuild/software/Anaconda3/4.0.0/lib/python3.5/site-packages/conda_build/skeletons/cpan.py", line 603, in get_release_info
    if (version is not None) and (version != LooseVersion('0') and
  File "/home/jillian/.local/easybuild/software/Anaconda3/4.0.0/lib/python3.5/distutils/version.py", line 46, in __eq__
    c = self._cmp(other)
  File "/home/jillian/.local/easybuild/software/Anaconda3/4.0.0/lib/python3.5/distutils/version.py", line 337, in _cmp
    if self.version < other.version:
TypeError: unorderable types: str() < int()
```

_Copied from original issue: conda/conda#3835_


Discussion:
@jerowe thanks for reporting.  Will fix for the next release.


Resolving patch:
diff --git a/conda_build/skeletons/cpan.py b/conda_build/skeletons/cpan.py
--- a/conda_build/skeletons/cpan.py
+++ b/conda_build/skeletons/cpan.py
@@ -600,7 +600,7 @@ def get_release_info(cpan_url, package, version, perl_version, config,
     # If the latest isn't the version we're looking for, we have to do another
     # request
     version_str = str(version)
-    if (version is not None) and (version != LooseVersion('0') and
+    if (version is not None) and (LooseVersion('0') != version_str and
             (rel_dict['version'] != version_str)):
         author = rel_dict['author']
         try:


Test patch:
Repository: conda/conda-build

Issue:
Bad symbolic links crash conda-build
We were getting this strange error in a CI build (package name redacted):
```
Copying /home/ubuntu/xxxxxx to /home/ubuntu/miniconda/conda-bld/conda-recipe_1478271856877/work
Traceback (most recent call last):
  File "/home/ubuntu/miniconda/lib/python3.5/site-packages/conda_build/utils.py", line 165, in copytree
    shutil.copy2(s, d)
  File "/home/ubuntu/miniconda/lib/python3.5/shutil.py", line 251, in copy2
    copyfile(src, dst, follow_symlinks=follow_symlinks)
  File "/home/ubuntu/miniconda/lib/python3.5/shutil.py", line 114, in copyfile
    with open(src, 'rb') as fsrc:
FileNotFoundError: [Errno 2] No such file or directory: '/home/ubuntu/xxxxxx/xxxxxx/bin/activate'
````
The file that is not found here is in fact a symbolic link: `/home/ubuntu/xxxxxx/xxxxxx/bin/activate` points to a path that does not exist in the CI instance. 

That entire `bin` directory had been checked into our repository unintentionally. The reason it got there [is a separate `conda` issue](https://github.com/conda/conda/issues/3808). However, its existence does not prevent our package from being built and tested _in situ_, so it seems to me that conda-build should be able to handle this error gracefully. 

I can't decide whether it should _ignore_ faulty symbolic links (or copy them without question), or whether it should do a better job of explaining the failure. But I don't think it should be responding in this manner; a more informative error message would be helpful, because it would help people diagnose the problem much more quickly.


Discussion:
OK.  I'll try to improve this for the next bugfix release.

Thank you! You might also be interested in the issue that created those symbolic links: https://github.com/conda/conda/issues/3808


Resolving patch:
diff --git a/conda_build/utils.py b/conda_build/utils.py
--- a/conda_build/utils.py
+++ b/conda_build/utils.py
@@ -110,6 +110,10 @@ def copy_into(src, dst, timeout=90, symlinks=False, lock=None):
             else:
                 src_folder = os.getcwd()
 
+        if os.path.islink(src) and not os.path.exists(os.path.realpath(src)):
+            log.warn('path %s is a broken symlink - ignoring copy', src)
+            return
+
         if not lock:
             lock = get_lock(src_folder, timeout=timeout)
         with lock:


Test patch:
Repository: conda/conda-build

Issue:
assertion error in script post.py
Hi,

I have the following error message when executing conda-build for recipe https://github.com/bioconda/bioconda-recipes/tree/master/recipes/parafly:

`Traceback (most recent call last):
  File "/tmp/mc2/bin/conda-build", line 6, in <module>
    sys.exit(conda_build.cli.main_build.main())
  File "/tmp/mc2/lib/python2.7/site-packages/conda_build/cli/main_build.py", line 242, in main
    execute(sys.argv[1:])
  File "/tmp/mc2/lib/python2.7/site-packages/conda_build/cli/main_build.py", line 234, in execute
    already_built=None, config=config)
  File "/tmp/mc2/lib/python2.7/site-packages/conda_build/api.py", line 77, in build
    need_source_download=need_source_download, config=config)
  File "/tmp/mc2/lib/python2.7/site-packages/conda_build/build.py", line 1099, in build_tree
    config=recipe_config)
  File "/tmp/mc2/lib/python2.7/site-packages/conda_build/build.py", line 791, in build
    croot=config.croot)
  File "/tmp/mc2/lib/python2.7/site-packages/conda_build/post.py", line 430, in post_build
    mk_relative(m, f, prefix)
  File "/tmp/mc2/lib/python2.7/site-packages/conda_build/post.py", line 389, in mk_relative
    mk_relative_linux(f, prefix=prefix, rpaths=m.get_value('build/rpaths', ['lib']))
  File "/tmp/mc2/lib/python2.7/site-packages/conda_build/post.py", line 362, in mk_relative_linux
    rel_ours, rel_stdlib, f, rpath)
AssertionError: utils.relative ./lib and relpath lib disagree for ParaFly, lib
`

Looks like the assertion `assert rel_ours == rel_stdlib` in post.py should pass but did not because of `./`
Am I right?

Loraine



Discussion:
seems linked to issue #1411

I have added `binary_relocation: False` the meta.yml like in https://github.com/bioconda/bioconda-recipes/pull/2686. It has solved the issue.
But not sure it is a good practice.

Thanks for posting that workaround.  We'll try to have a proper fix soon.


Resolving patch:
diff --git a/conda_build/post.py b/conda_build/post.py
--- a/conda_build/post.py
+++ b/conda_build/post.py
@@ -364,8 +364,8 @@ def mk_relative_linux(f, prefix, rpaths=('lib',)):
             # IMHO utils.relative shouldn't exist, but I am too paranoid to remove
             # it, so instead, make sure that what I think it should be replaced by
             # gives the same result and assert if not. Yeah, I am a chicken.
-            rel_ours = utils.relative(f, rpath)
-            rel_stdlib = os.path.relpath(rpath, os.path.dirname(f))
+            rel_ours = os.path.normpath(utils.relative(f, rpath))
+            rel_stdlib = os.path.normpath(os.path.relpath(rpath, os.path.dirname(f)))
             assert rel_ours == rel_stdlib, \
                 'utils.relative {0} and relpath {1} disagree for {2}, {3}'.format(
                 rel_ours, rel_stdlib, f, rpath)


Test patch:
Repository: conda/conda-build

Issue:
conda build --check fails when using source_files directive
Using conda-build 2.0.6
Reproduce by creating a recipe with a source_files directive

```
conda` build --check recipe
Traceback (most recent call last):
  File "REDACTED/miniconda2/bin/conda-build", line 6, in <module>
    sys.exit(conda_build.cli.main_build.main())
  File "REDACTED/miniconda2/lib/python2.7/site-packages/conda_build/cli/main_build.py", line 242, in main
    execute(sys.argv[1:])
  File "REDACTED/miniconda2/lib/python2.7/site-packages/conda_build/cli/main_build.py", line 227, in execute
    action(m, config)
  File "REDACTED/miniconda2/lib/python2.7/site-packages/conda_build/cli/main_build.py", line 181, in check_action
    return api.check(metadata.path, config=config)
  File "REDACTED/miniconda2/lib/python2.7/site-packages/conda_build/api.py", line 62, in check
    return metadata.check_fields()
  File "REDACTED/miniconda2/lib/python2.7/site-packages/conda_build/metadata.py", line 536, in check_fields
    (section, key))
ValueError: in section u'test': unknown key u'source_files'
```



Discussion:
Sorry for the delay here.  Have you tried the latest conda-build (2.0.10)?  Is this still an issue?  FWIW, we use this for conda-build itself, so it should be reasonably well tested.
Yes this is still an issue as of 
```
$ conda build -V
conda-build 2.0.11
```
OK, thanks.  I will investigate this before the next bug-fix release.

Resolving patch:
diff --git a/conda_build/metadata.py b/conda_build/metadata.py
--- a/conda_build/metadata.py
+++ b/conda_build/metadata.py
@@ -296,7 +296,7 @@ def _git_clean(source_meta):
     'requirements': ['build', 'run', 'conflicts'],
     'app': ['entry', 'icon', 'summary', 'type', 'cli_opts',
             'own_environment'],
-    'test': ['requires', 'commands', 'files', 'imports'],
+    'test': ['requires', 'commands', 'files', 'imports', 'source_files'],
     'about': ['home', 'dev_url', 'doc_url', 'license_url',  # these are URLs
               'license', 'summary', 'description', 'license_family',  # text
               'license_file', 'readme',  # paths in source tree


Test patch:
Repository: conda/conda-build

Issue:
Python code compiled when it shouldn't be
Ran into an issue earlier, which was correctly identified thanks to @msarahan . It turns out `conda-build` was compiling Python code even though Python was not in `requirements/build`.

xref: https://github.com/conda-forge/git-feedstock/pull/20#issuecomment-267858227


Discussion:

Resolving patch:
diff --git a/conda_build/post.py b/conda_build/post.py
--- a/conda_build/post.py
+++ b/conda_build/post.py
@@ -176,6 +176,8 @@ def rm_pyc(files, prefix):
 
 
 def compile_missing_pyc(files, cwd, python_exe, skip_compile_pyc=()):
+    if not os.path.isfile(python_exe):
+        return
     compile_files = []
     skip_compile_pyc_n = [os.path.normpath(skip) for skip in skip_compile_pyc]
     skipped_files = set()


Test patch:
Repository: conda/conda-build

Issue:
TypeError: unorderable types: int() < str() on cpan skeleton
```
➜  perl-dist-milla-all conda skeleton cpan --recursive Dist::Milla
Using url https://cpan.metacpan.org/authors/id/M/MI/MIYAGAWA/Dist-Milla-v1.0.17.tar.gz (22897) for Dist::Milla.
Processing dependencies for Dist::Milla....Traceback (most recent call last):
  File "/home/jillian/.local/easybuild/software/Anaconda3/4.0.0/bin/conda-skeleton", line 6, in <module>
    sys.exit(conda_build.cli.main_skeleton.main())
  File "/home/jillian/.local/easybuild/software/Anaconda3/4.0.0/lib/python3.5/site-packages/conda_build/cli/main_skeleton.py", line 65, in main
    return execute(sys.argv[1:])
  File "/home/jillian/.local/easybuild/software/Anaconda3/4.0.0/lib/python3.5/site-packages/conda_build/cli/main_skeleton.py", line 61, in execute
    api.skeletonize(package, args.repo, config=config)
  File "/home/jillian/.local/easybuild/software/Anaconda3/4.0.0/lib/python3.5/site-packages/conda_build/api.py", line 189, in skeletonize
    recursive=recursive, config=config, **kwargs)
  File "/home/jillian/.local/easybuild/software/Anaconda3/4.0.0/lib/python3.5/site-packages/conda_build/skeletons/cpan.py", line 301, in skeletonize
    recursive=recursive, config=config)
  File "/home/jillian/.local/easybuild/software/Anaconda3/4.0.0/lib/python3.5/site-packages/conda_build/skeletons/cpan.py", line 493, in deps_for_package
    dep_version, perl_version, dependency=True, config=config)
  File "/home/jillian/.local/easybuild/software/Anaconda3/4.0.0/lib/python3.5/site-packages/conda_build/skeletons/cpan.py", line 603, in get_release_info
    if (version is not None) and (LooseVersion('0') != version_str and
  File "/home/jillian/.local/easybuild/software/Anaconda3/4.0.0/lib/python3.5/distutils/version.py", line 46, in __eq__
    c = self._cmp(other)
  File "/home/jillian/.local/easybuild/software/Anaconda3/4.0.0/lib/python3.5/distutils/version.py", line 337, in _cmp
    if self.version < other.version:
TypeError: unorderable types: int() < str()
```


Discussion:

Resolving patch:
diff --git a/conda_build/skeletons/cpan.py b/conda_build/skeletons/cpan.py
--- a/conda_build/skeletons/cpan.py
+++ b/conda_build/skeletons/cpan.py
@@ -600,8 +600,8 @@ def get_release_info(cpan_url, package, version, perl_version, config,
     # If the latest isn't the version we're looking for, we have to do another
     # request
     version_str = str(version)
-    if (version is not None) and (LooseVersion('0') != version_str and
-            (rel_dict['version'] != version_str)):
+    if (version is not None) and (LooseVersion('0') != LooseVersion(version_str) and
+            (LooseVersion(rel_dict['version']) != LooseVersion(version_str))):
         author = rel_dict['author']
         try:
             new_rel_dict = get_cpan_api_url('{0}/v0/release/{1}/{2}-{3}'.format(cpan_url,


Test patch:
Repository: conda/conda-build

Issue:
Encoding issues on Windows build in the fix_staged_scripts step
Hello.

I'm trying to build the [waf package on windows](https://github.com/conda-forge/waf-feedstock/pull/4), but I'm having some trouble with ```conda_build``` in the ```fix_staged_scripts``` step. I couldn't find any similar issue. I'll follow with some details:

While building, there is a file named ```waf``` that is meant to be moved to the ```%SCRIPTS%``` folder. The problem seems to be that ```conda-build``` tries to read the file to do some checking, but fails to do so because of the file encoding (https://github.com/conda/conda-build/blob/master/conda_build/windows.py#L45)

I believe the waf file should be opened with encoding='iso-8859-1'. Making a local change replacing with the code bellow builds the package successfully, but is not a solution:

```
with open(join(scripts_dir, fn), encoding='iso-8859-1') as f:
```

Any suggestions here? I'm unsure how this case should be handled.
I'll leave some suggestions that occurred me:

1- Fix this at package level, by avoiding running into the ```fix_staged_scripts``` function, somehow. (I don't like this solution because it'll not really fix anything besides this specific package)

2- Do some try...catch with several encodings near the presented code above

3- Have a metadata somewhere (maybe in the recipe?) to handle the file encoding. Fallback to the current case if none.


Discussion:
Thanks for isolating this.  I have had a very hard time reproducing it, but your case here should work.

Resolving patch:
diff --git a/conda_build/windows.py b/conda_build/windows.py
--- a/conda_build/windows.py
+++ b/conda_build/windows.py
@@ -4,6 +4,7 @@
 import sys
 from os.path import isdir, join, dirname, isfile
 
+import bs4
 # importing setuptools patches distutils so that it knows how to find VC for python 2.7
 import setuptools  # noqa
 # Leverage the hard work done by setuptools/distutils to find vcvarsall using
@@ -43,7 +44,7 @@ def fix_staged_scripts(scripts_dir):
             continue
 
         with open(join(scripts_dir, fn)) as f:
-            line = f.readline().lower()
+            line = bs4.UnicodeDammit(f.readline()).unicode_markup.lower()
             # If it's a #!python script
             if not (line.startswith('#!') and 'python' in line.lower()):
                 continue


Test patch:
Repository: conda/conda-build

Issue:
directories like xxx.git are removed from tarball
I'm trying to build a conda package for [craftr](https://craftr.net/), which contains a directory `craftr/stl/craftr.utils.git/` that is filtered out from the tarball during build. The problem is then detected in `tarcheck.py`. See https://github.com/conda-forge/staged-recipes/pull/2189

I think it comes from the [filter_files](https://github.com/conda/conda-build/blob/2.1.0/conda_build/build.py#L782-L788) function.

I'm working on a PR.


Discussion:
Thanks. Naive of me to think that filter files would always work...

Looking forward to the PR.

Resolving patch:
diff --git a/conda_build/build.py b/conda_build/build.py
--- a/conda_build/build.py
+++ b/conda_build/build.py
@@ -781,7 +781,7 @@ def warn_on_old_conda_build(index=None, installed_version=None, available_packag
 """ % (installed_version, available_packages[-1]), file=sys.stderr)
 
 
-def filter_files(files_list, prefix, filter_patterns=('.*[\\\\/]?\.git[\\\\/].*', )):
+def filter_files(files_list, prefix, filter_patterns=('(.*[\\\\/])?\.git[\\\\/].*', )):
     """Remove things like .git from the list of files to be copied"""
     for pattern in filter_patterns:
         r = re.compile(pattern)


Test patch:
diff --git a/tests/test_build.py b/tests/test_build.py
--- a/tests/test_build.py
+++ b/tests/test_build.py
@@ -295,5 +295,13 @@ def test_create_info_files_json_no_inodes(testing_workdir, test_metadata):
 
 
 def test_filter_files():
-    files_list = ['a', '.git/a', 'something/.git/a', '.git\\a', 'something\\.git\\a']
-    assert build.filter_files(files_list, '') == ['a']
+    # Files that should be filtered out.
+    files_list = ['.git/a', 'something/.git/a', '.git\\a', 'something\\.git\\a']
+    assert not build.filter_files(files_list, '')
+
+    # Files that should *not* be filtered out.
+    # Example of valid 'x.git' directory:
+    #    lib/python3.4/site-packages/craftr/stl/craftr.utils.git/Craftrfile
+    files_list = ['a', 'x.git/a', 'something/x.git/a',
+                  'x.git\\a', 'something\\x.git\\a']
+    assert len(build.filter_files(files_list, '')) == len(files_list)
Repository: conda/conda-build

Issue:
'NoPackagesFoundError' object has no attribute 'packages'
_From @danfischetti on January 26, 2017 18:42_

I'm running conda 4.3.8 with conda-build 2.1.2. and I get this error when I try to build a package with unsatisfiable dependencies. conda-build/build.py:1521 tries to access the 'packages' attribute of the NoPackagesFoundError but in https://github.com/conda/conda/blob/master/conda/exceptions.py the class is defined with the attribute 'pkgs'

_Copied from original issue: conda/conda#4437_


Discussion:

Resolving patch:
diff --git a/conda_build/build.py b/conda_build/build.py
--- a/conda_build/build.py
+++ b/conda_build/build.py
@@ -1518,7 +1518,7 @@ def build_tree(recipe_list, config, build_only=False, post=False, notest=False,
                         add_recipes.append(recipe_dir)
                 else:
                     raise RuntimeError("Can't build {0} due to unsatisfiable dependencies:\n{1}"
-                                       .format(recipe, e.packages) + "\n\n" + extra_help)
+                                       .format(recipe, e.pkgs) + "\n\n" + extra_help)
             recipe_list.extendleft(add_recipes)
 
         # outputs message, or does upload, depending on value of args.anaconda_upload


Test patch:
Repository: conda/conda-build

Issue:
Relative path in source causes rm_rf to remove linked source files
from https://github.com/conda/conda-build/pull/1598#issuecomment-272691858

@msarahan this commit (specifically these lines as found by running git bisect) - causes some odd behaviour - seen here. It seems that if you have a script in your project then the rm_rf call traverses the possibly? sym-linked script file and thus deletes the script in the original repository. This also may only be an issue for recipes built where the source is next to the recipe folder and thus copied using a relative path:

source:
  path: ../

To be clear:

After this commit, after the test phase the bin/lsfm script is deleted from inside the repository and not just from the work/build environment.


Discussion:
Thanks for adding as an issue - didn't raise in case something I did was wrong 😄 
@msarahan saw your request for feedback if you had caught the issue with #1684, I'm afraid I'm still seeing the same behaviour of scripts being deleted on conda build 2.1.2:

https://travis-ci.org/menpo/workerbee/jobs/195511033#L520

I didn't follow @patricksnape's git bisect down into the weeds to be able to help much on how to refine the patch, but it does seem to still be an issue we are seeing.
Thanks @jabooth.  Using your recipe will help give me something of a start.

Resolving patch:
diff --git a/conda_build/build.py b/conda_build/build.py
--- a/conda_build/build.py
+++ b/conda_build/build.py
@@ -881,6 +881,8 @@ def order(f):
     # remove files from build prefix.  This is so that they can be included in other packages.  If
     #     we were to leave them in place, then later scripts meant to also include them may not.
     for f in files:
+        if not os.path.isabs(f):
+            f = os.path.abspath(os.path.normpath(os.path.join(metadata.config.build_prefix, f)))
         utils.rm_rf(f)
     return final_output
 


Test patch:
diff --git a/tests/test_subpackages.py b/tests/test_subpackages.py
--- a/tests/test_subpackages.py
+++ b/tests/test_subpackages.py
@@ -21,12 +21,14 @@ def test_autodetect_raises_on_invalid_extension(test_config):
         api.build(os.path.join(subpackage_dir, '_invalid_script_extension'), config=test_config)
 
 
-def test_rm_rf_does_not_follow_links(test_config):
+# regression test for https://github.com/conda/conda-build/issues/1661
+def test_rm_rf_does_not_remove_relative_source_package_files(test_config, monkeypatch):
     recipe_dir = os.path.join(subpackage_dir, '_rm_rf_stays_within_prefix')
-    bin_file_that_disappears = os.path.join(recipe_dir, 'bin', 'lsfm')
+    monkeypatch.chdir(recipe_dir)
+    bin_file_that_disappears = os.path.join('bin', 'lsfm')
     if not os.path.isfile(bin_file_that_disappears):
         with open(bin_file_that_disappears, 'w') as f:
             f.write('weee')
     assert os.path.isfile(bin_file_that_disappears)
-    api.build(os.path.join(recipe_dir, 'conda'), config=test_config)
+    api.build('conda', config=test_config)
     assert os.path.isfile(bin_file_that_disappears)