internlm/SWE-Fixer-Train-110K huggingface dataset:
Repository: conda/conda-build

Issue:
requirements.txt should not override requirements in meta.yaml
The change in #190 blew up in my face today because I have a package that I want to be installable both via `pip` and via `conda`.  Calling the pip requirements file `requirements.txt` has been standard practice for some time, and I had no idea conda-build was reading it now.

Anyway, what happened was that I had `beautifulsoup4` in my `requirements.txt`, which is the name of the package on PyPI, whereas the conda package is called `beautiful-soup`.  The build test kept failing saying it couldn't find a conda package called `beautifulsoup4` and I was perplexed, because my requirements section in my `meta.yaml` clearly reads:

``` yaml
requirements:
  build:
    - python
    - regex
    - setuptools
    - beautiful-soup
    - six

  run:
    - python
    - regex
    - setuptools
    - beautiful-soup
    - six
```

but if you look in the `index.json` that gets generated, you see:

``` python
{
  ...
  "depends": [
    "beautifulsoup4",
    "regex",
    "six"
  ],
  ...
}
```

Overriding what people actually put in `meta.yaml` seems highly undesirable to me, and if you want to allow it at all, it should be optional (and not on by default).



Discussion:

Resolving patch:
diff --git a/conda_build/metadata.py b/conda_build/metadata.py
--- a/conda_build/metadata.py
+++ b/conda_build/metadata.py
@@ -258,7 +258,7 @@ def parse_again(self):
             return
         self.meta = parse(get_contents(self.meta_path))
 
-        if isfile(self.requirements_path):
+        if isfile(self.requirements_path) and not self.meta['requirements']['run']:
             self.meta.setdefault('requirements', {})
             run_requirements = specs_from_url(self.requirements_path)
             self.meta['requirements']['run'] = run_requirements


Test patch:
Repository: conda/conda-build

Issue:
Update windows.py to find the free Microsoft Visual C++ Compiler for Python 2.7
The  [Microsoft Visual C++ Compiler for Python 2.7](http://www.microsoft.com/en-us/download/details.aspx?id=44266) is a free light weight environment for building binary windows packages, but [windows.py](https://github.com/conda/conda-build/blob/c4e21e84b7fb7c1a9646ba179a73c3075a83460d/conda_build/windows.py#L56-L77) is not finding the `vcvarsall.bat` file.  

I've got Anaconda 64 and when I try to build, I get:

```
 raise DistutilsPlatformError("Unable to find vcvarsall.bat")
distutils.errors.DistutilsPlatformError: Unable to find vcvarsall.bat
```

On my Windows 7 system,  the `vcvarsall.bat` file got installed at `C:\Users\rsignell\AppData\Local\Programs\Common\Microsoft\Visual C++ for Python\9.0\vcvarsall.bat` and from my Anaconda 64 command prompt, we can use the `%localappdata%` environment variable:

![2015-01-27_1757](https://cloud.githubusercontent.com/assets/1872600/5929025/09da3714-a64e-11e4-932a-ffedb69ece34.png)



Discussion:

Resolving patch:
diff --git a/conda_build/windows.py b/conda_build/windows.py
--- a/conda_build/windows.py
+++ b/conda_build/windows.py
@@ -59,6 +59,8 @@ def msvc_env_cmd():
     else:
         program_files = os.environ['ProgramFiles']
 
+    localappdata = os.environ.get("localappdata")
+
     if config.PY3K:
         vcvarsall = os.path.join(program_files,
                                  r'Microsoft Visual Studio 10.0'
@@ -68,6 +70,13 @@ def msvc_env_cmd():
                                  r'Microsoft Visual Studio 9.0'
                                  r'\VC\vcvarsall.bat')
 
+    # Try the Microsoft Visual C++ Compiler for Python 2.7
+    if not isfile(vcvarsall) and localappdata and not config.PY3K:
+        vcvarsall = os.path.join(localappdata, "Programs", "Common",
+            "Microsoft", "Visual C++ for Python", "9.0", "vcvarsall.bat")
+    if not isfile(vcvarsall) and program_files and not config.PY3K:
+        vcvarsall = os.path.join(program_files, 'Common Files',
+            'Microsoft', 'Visual C++ for Python', "9.0", "vcvarsall.bat")
     if not isfile(vcvarsall):
         print("Warning: Couldn't find Visual Studio: %r" % vcvarsall)
         return ''


Test patch:
Repository: conda/conda-build

Issue:
Conda-build should allow inclusion of a readme in the metadata
Binstar.org supports uploading a README, however conda does not allow for including a README. 
Can we add a `readme` tag in the `meta.yaml` to include a `README.*` in the info?



Discussion:
The README on Binstar is package level, not file level, as far as I can tell. How will it handle the readmes coming from multiple files of the same package?

binstar.org shows the README from the latest version. The command line client only uploads a README **once** for each version.

Does the command line client already expect it to be `info/README.*` in the package?

I will need to modify the client to look for it after conda includes it. So you can include it anywhere you like. 

OK. I'll think I'll do what I described. I don't want to put it in the json as that could get too big for the repodata. 

Another question, does Binstar actually support multiple markup formats? 

It supports rst and markdown


Resolving patch:
diff --git a/conda_build/build.py b/conda_build/build.py
--- a/conda_build/build.py
+++ b/conda_build/build.py
@@ -156,6 +156,17 @@ def create_info_files(m, files, include_recipe=True):
     with open(join(recipe_dir, 'meta.yaml'), 'w') as fo:
         yaml.safe_dump(m.meta, fo)
 
+    readme = m.get_value('about/readme')
+    if readme:
+        src = join(source.get_dir(), readme)
+        if not os.path.exists(src):
+            sys.exit("Error: Could not find the readme: %s" % readme)
+        dst = join(config.info_dir, readme)
+        shutil.copy(src, dst)
+        if os.path.split(readme)[1] not in {"README.md", "README.rst", "README"}:
+            print("WARNING: Binstar only recognizes about/readme as README.md and README.rst",
+                file=sys.stderr)
+
     # Deal with Python 2 and 3's different json module type reqs
     mode_dict = {'mode': 'w', 'encoding': 'utf-8'} if PY3 else {'mode': 'wb'}
     with open(join(config.info_dir, 'index.json'), **mode_dict) as fo:
diff --git a/conda_build/metadata.py b/conda_build/metadata.py
--- a/conda_build/metadata.py
+++ b/conda_build/metadata.py
@@ -193,7 +193,7 @@ def _git_clean(source_meta):
     'app': ['entry', 'icon', 'summary', 'type', 'cli_opts',
             'own_environment'],
     'test': ['requires', 'commands', 'files', 'imports'],
-    'about': ['home', 'license', 'summary'],
+    'about': ['home', 'license', 'summary', 'readme'],
 }
 
 


Test patch:
Repository: conda/conda-build

Issue:
Issue with tests when building a package for Python
If you build a package for Python, it wants to run the tests using Python, which means that it tries to install the version of Python that conda is installed into into the test environment. If this differs from the Python version of the package, it fails because the packages conflict. 



Discussion:
I guess it really uses the `CONDA_PY` version, which defaults to the version conda is installed into. So when building a package for `python`, `CONDA_PY` should be changed to match that version. 


Resolving patch:
diff --git a/conda_build/build.py b/conda_build/build.py
--- a/conda_build/build.py
+++ b/conda_build/build.py
@@ -458,15 +458,19 @@ def test(m, verbose=True, channel_urls=(), override_channels=False):
     rm_rf(config.test_prefix)
     specs = ['%s %s %s' % (m.name(), m.version(), m.build_id())]
 
-    if py_files:
+    # add packages listed in test/requires
+    specs_include_python = False
+    for spec in m.get_value('test/requires', []):
+        specs.append(spec)
+        if spec.startswith('python ') or spec == 'python':
+            specs_include_python = True
+
+    if py_files and not specs_include_python:
         # as the tests are run by python, we need to specify it
         specs += ['python %s*' % environ.get_py_ver()]
     if pl_files:
         # as the tests are run by perl, we need to specify it
         specs += ['perl %s*' % environ.get_perl_ver()]
-    # add packages listed in test/requires
-    for spec in m.get_value('test/requires', []):
-        specs.append(spec)
 
     create_env(config.test_prefix, specs, verbose=verbose,
         channel_urls=channel_urls, override_channels=override_channels)


Test patch:
Repository: conda/conda-build

Issue:
MD5 checking argument not passed to update_index
I was happy to see that there was a `-c` argument to `conda index` which forces it to use md5 hashes instead of file modification times. However, looks like `main_index.py` never passes that argument on to the `update_index()` function, i.e., 

``` python
...
update_index(path, verbose=(not args.quiet), force=args.force)
...
```

should actually be:

``` python
...
update_index(path, verbose=(not args.quiet), force=args.force, check_md5=args.check_md5)
...
```



Discussion:
@dan-blanchard 


Resolving patch:
diff --git a/conda_build/main_index.py b/conda_build/main_index.py
--- a/conda_build/main_index.py
+++ b/conda_build/main_index.py
@@ -40,7 +40,7 @@ def main():
         dir_paths = [path.decode(getpreferredencoding()) for path in dir_paths]
 
     for path in dir_paths:
-        update_index(path, verbose=(not args.quiet), force=args.force)
+        update_index(path, verbose=(not args.quiet), force=args.force, check_md5=args.check_md5)
 
 
 if __name__ == '__main__':


Test patch:
Repository: conda/conda-build

Issue:
Noarch packages are placed into the wrong directory on build
When I build a noarch package it still gets placed in the `osx-64` subdirectory.

```
# If you want to upload this package to binstar.org later, type:
#
# $ binstar upload /Users/sean/anaconda/conda-bld/osx-64/flask-0.10-py_1.tar.bz2
#
# To have conda build upload to binstar automatically, use
# $ conda config --set binstar_upload yes
```

`/Users/sean/anaconda/conda-bld/osx-64/flask-0.10-py_1.tar.bz2` should be `/Users/sean/anaconda/conda-bld/noarch/flask-0.10-py_1.tar.bz2`



Discussion:
@ilanschnell @asmeurer I'm also facing the same issue, any hints on solving it?


Resolving patch:
diff --git a/conda_build/build.py b/conda_build/build.py
--- a/conda_build/build.py
+++ b/conda_build/build.py
@@ -313,6 +313,9 @@ def build(m, get_src=True, verbose=True, post=None, channel_urls=(), override_ch
     :type post: bool or None. None means run the whole build. True means run
     post only. False means stop just before the post.
     '''
+    if m.get_value('build/noarch_python'):
+        config.noarch = True
+
     if (m.get_value('build/detect_binary_files_with_prefix')
         or m.binary_has_prefix_files()):
         # We must use a long prefix here as the package will only be
diff --git a/conda_build/config.py b/conda_build/config.py
--- a/conda_build/config.py
+++ b/conda_build/config.py
@@ -26,6 +26,8 @@ class Config(object):
 
     PY3K = int(bool(CONDA_PY >= 30))
 
+    noarch = False
+
     def get_conda_py(self):
         return self.CONDA_PY
 
@@ -95,7 +97,10 @@ def broken_dir(self):
 
     @property
     def bldpkgs_dir(self):
-        return join(self.croot, cc.subdir)
+        if self.noarch:
+            return join(self.croot, "noarch")
+        else:
+            return join(self.croot, cc.subdir)
 
 config = Config()
 


Test patch:
Repository: conda/conda-build

Issue:
conda pipbuild fails to fetch metadata for some packages
It appears that `conda pipbuild` fails when fetching metadata for some packages. For instance, here is an example with `six` below. Note after attempting to fetch metadata, it simply drops back to prompt.

```
$ conda pipbuild six
Package search: {'_pypi_ordering': 20, 'version': '1.9.0', 'name': 'six', 'summary': 'Python 2 and 3 compatibility utilities'}
Fetching package metadata: ......
$
```



Discussion:
FWIW, this still fails in 1.13.0.

It is likely that this behaviour is because the binary for package `six` already exists, for such cases `pipbuild` exists without any message, see https://github.com/conda/conda-build/issues/435.
You can verify that by printing a message after this line
https://github.com/conda/conda-build/blob/master/conda_build/main_pipbuild.py#L297

Interesting, you may very well be right. If that's true, then I see the following problems.
1. There is no error message provided.
2. This disallows building a newer version of a package that is already present.


Resolving patch:
diff --git a/conda_build/main_pipbuild.py b/conda_build/main_pipbuild.py
--- a/conda_build/main_pipbuild.py
+++ b/conda_build/main_pipbuild.py
@@ -99,29 +99,6 @@ def handle_binstar_upload(path):
 # Modify the recipe directory to make a new recipe with just the dependencies
 #   and a build script that says pip install for both build.sh and build.bat
 
-
-def conda_package_exists(pkgname, version=None):
-    from conda.api import get_index
-    from conda.resolve import MatchSpec, Resolve
-
-    pyver = 'py%s' % sys.version[:3].replace('.', '')
-    index = get_index(use_cache=True)
-    r = Resolve(index)
-    try:
-        pkgs = r.get_pkgs(MatchSpec(pkgname))
-    except RuntimeError:
-        return False
-    exists = False
-    for pkg in pkgs:
-        match_pyver = pkg.build.startswith(pyver)
-        if not match_pyver:
-            continue
-        if version and pkg.version != version:
-            continue
-        exists = True
-        break
-    return exists
-
 skeleton_template = "conda skeleton pypi {0} --no-prompt"
 skeleton_template_wversion = "conda skeleton pypi {0} --version {1} --no-prompt"
 build_template = "conda build {0} --no-binstar-upload --no-test"
@@ -316,8 +293,6 @@ def make_recipe(package, version, noarch_python=False):
 
 
 def build_package(package, version=None, noarch_python=False):
-    if conda_package_exists(package):
-        return 0
     if ' ' in package:
         package, version = package.split(' ')
     try:


Test patch:
Repository: conda/conda-build

Issue:
Error building when previous build was stoped.
Trying to build a package whose previous build was canceled halfway. 

The solution is to remove the previous tar (corrupted) that is obtained with `conda build --output .`
There should be a nice error message, requesting the deletion .

```
 File "/Users/pedro/anaconda/lib/python2.7/site-packages/conda_build/build.py", line 339, in build
    override_channels=override_channels)
  File "/Users/pedro/anaconda/lib/python2.7/site-packages/conda_build/build.py", line 244, in create_env
    update_index(config.bldpkgs_dir)
  File "/Users/pedro/anaconda/lib/python2.7/site-packages/conda_build/index.py", line 87, in update_index
    d = read_index_tar(path)
  File "/Users/pedro/anaconda/lib/python2.7/site-packages/conda_build/index.py", line 21, in read_index_tar
    with tarfile.open(tar_path) as t:
  File "/Users/pedro/anaconda/lib/python2.7/tarfile.py", line 1672, in open
    raise ReadError("file could not be opened successfully")
tarfile.ReadError: file could not be opened successfully
```

```
Current conda install:

             platform : osx-64
        conda version : 3.14.0


conda-build version : 1.14.0
       python version : 2.7.10.final.0
     requests version : 2.7.0
     root environment : /Users/pedro/anaconda  (writable)
  default environment : /Users/pedro/anaconda
     envs directories : /Users/pedro/anaconda/envs
        package cache : /Users/pedro/anaconda/pkgs
         channel URLs : https://repo.continuum.io/pkgs/free/osx-64/
                        https://repo.continuum.io/pkgs/free/noarch/
                        https://repo.continuum.io/pkgs/pro/osx-64/
                        https://repo.continuum.io/pkgs/pro/noarch/
          config file : None
    is foreign system : False
```



Discussion:

Resolving patch:
diff --git a/conda_build/index.py b/conda_build/index.py
--- a/conda_build/index.py
+++ b/conda_build/index.py
@@ -18,14 +18,17 @@
 
 def read_index_tar(tar_path):
     """ Returns the index.json dict inside the given package tarball. """
-    with tarfile.open(tar_path) as t:
-        try:
-            return json.loads(t.extractfile('info/index.json').read().decode('utf-8'))
-        except EOFError:
-            raise RuntimeError("Could not extract %s. File probably corrupt."
-                % tar_path)
-        except OSError as e:
-            raise RuntimeError("Could not extract %s (%s)" % (tar_path, e))
+    try:
+        with tarfile.open(tar_path) as t:
+            try:
+                return json.loads(t.extractfile('info/index.json').read().decode('utf-8'))
+            except EOFError:
+                raise RuntimeError("Could not extract %s. File probably corrupt."
+                    % tar_path)
+            except OSError as e:
+                raise RuntimeError("Could not extract %s (%s)" % (tar_path, e))
+    except tarfile.ReadError:
+        raise RuntimeError("Could not extract metadata from %s. File probably corrupt." % tar_path)
 
 def write_repodata(repodata, dir_path):
     """ Write updated repodata.json and repodata.json.bz2 """


Test patch:
Repository: conda/conda-build

Issue:
conda build --output doesn't honor noarch_python
I'm using the recent feature of `noarch_python: true` in meta.yaml (merged in #317). My package builds fine, but when I try to get the package path via `conda build --output conda_build.recipe`, it shows the path with `linux-64` instead of `noarch`, where the package is stored after successful build.

```
$ conda build conda_build.recipe
...
# If you want to upload this package to binstar.org later, type:
#
# $ binstar upload /home/jenkins/conda-bld/noarch/mypackage-1.0-py_27.tar.bz2
```

```
$ conda build --output conda_build.recipe
/home/jenkins/conda-bld/linux-64/mypackage-1.0-py_27.tar.bz2
```



Discussion:
@ilanschnell can it be fixed quickly? Any workaround (except `sed 's/linux-64/noarch/'`)?

See also: #443 


Resolving patch:
diff --git a/conda_build/build.py b/conda_build/build.py
--- a/conda_build/build.py
+++ b/conda_build/build.py
@@ -311,8 +311,6 @@ def build(m, get_src=True, verbose=True, post=None, channel_urls=(), override_ch
     :type post: bool or None. None means run the whole build. True means run
     post only. False means stop just before the post.
     '''
-    if m.get_value('build/noarch_python'):
-        config.noarch = True
 
     if (m.get_value('build/detect_binary_files_with_prefix')
         or m.binary_has_prefix_files()):
diff --git a/conda_build/main_build.py b/conda_build/main_build.py
--- a/conda_build/main_build.py
+++ b/conda_build/main_build.py
@@ -271,6 +271,8 @@ def execute(args, parser):
 
             try:
                 m = MetaData(recipe_dir)
+                if m.get_value('build/noarch_python'):
+                    config.noarch = True
             except exceptions.YamlParsingError as e:
                 sys.stderr.write(e.error_msg())
                 sys.exit(1)


Test patch:
Repository: conda/conda-build

Issue:
AppVeyor: Commit message with braces -> failed build
https://ci.appveyor.com/project/mpi4py/mpi4py/build/2.0.0a0-13/job/0q0w2g5o32qk3m94#L522

PS: I got a warning about conda-build being outdated. Isn't `conda update --all` supposed to update it? Maybe conflicting versions with dependencies?



Discussion:
I don't understand why update --all didn't seem to update anything. 

Well, the thing is that you don't see the output just because the way the PowerShell script invokes conda. I've added an explicit conda update conda-build, now things seems to be working. However, the issue with commit messages and braces is probably still there.

I guess we need to quote the environment variables better. 


Resolving patch:
diff --git a/conda_build/windows.py b/conda_build/windows.py
--- a/conda_build/windows.py
+++ b/conda_build/windows.py
@@ -116,7 +116,7 @@ def build(m):
         with open(join(src_dir, 'bld.bat'), 'w') as fo:
             fo.write(msvc_env_cmd())
             for kv in iteritems(env):
-                fo.write('set %s=%s\n' % kv)
+                fo.write('set "%s=%s"\n' % kv)
             # more debuggable with echo on
             fo.write('@echo on\n')
             fo.write("REM ===== end generated header =====\n")


Test patch:
Repository: conda/conda-build

Issue:
Ability to not include recipe in package
conda-build always includes the recipe in the conda package that is created.  There should be the ability to not do that, eg. an option `--no-recipe-include`.



Discussion:
Should it be a command line flag or a meta.yaml option? 

I think this should not be in the `meta.yaml`, it should be an option like `--no-recipe-include`.

This should be very easy to fix. The flag is already there internally. 

Yes, I saw that, https://github.com/conda/conda-build/blob/master/conda_build/build.py#L442
When is `bool(m.path)` ever not `True`?

In `conda metapackage`. It creates a Metadata object without a physical recipe. 

Ahh, I see.


Resolving patch:
diff --git a/conda_build/build.py b/conda_build/build.py
--- a/conda_build/build.py
+++ b/conda_build/build.py
@@ -307,7 +307,8 @@ def bldpkg_path(m):
     '''
     return join(config.bldpkgs_dir, '%s.tar.bz2' % m.dist())
 
-def build(m, get_src=True, verbose=True, post=None, channel_urls=(), override_channels=False):
+def build(m, get_src=True, verbose=True, post=None, channel_urls=(),
+    override_channels=False, include_recipe=True):
     '''
     Build the package with the specified metadata.
 
@@ -439,7 +440,7 @@ def build(m, get_src=True, verbose=True, post=None, channel_urls=(), override_ch
                     join(config.build_prefix, f)),)))
         post_build(m, sorted(files2 - files1))
         create_info_files(m, sorted(files2 - files1),
-                          include_recipe=bool(m.path))
+                          include_recipe=bool(m.path) and include_recipe)
         if m.get_value('build/noarch_python'):
             import conda_build.noarch_python as noarch_python
             noarch_python.transform(m, sorted(files2 - files1))
diff --git a/conda_build/main_build.py b/conda_build/main_build.py
--- a/conda_build/main_build.py
+++ b/conda_build/main_build.py
@@ -89,6 +89,13 @@ def main():
         dest='binstar_upload',
         default=config.binstar_upload,
     )
+    p.add_argument(
+        "--no-include-recipe",
+        action="store_false",
+        help="Don't include the recipe inside the built package.",
+        dest='include_recipe',
+        default=True,
+    )
     p.add_argument(
         "--output",
         action="store_true",
@@ -372,7 +379,8 @@ def execute(args, parser):
                     post = None
                 try:
                     build.build(m, verbose=not args.quiet, post=post,
-                        channel_urls=channel_urls, override_channels=args.override_channels)
+                        channel_urls=channel_urls,
+                        override_channels=args.override_channels, include_recipe=args.include_recipe)
                 except RuntimeError as e:
                     error_str = str(e)
                     if error_str.startswith('No packages found') or error_str.startswith('Could not find some'):


Test patch:
Repository: conda/conda-build

Issue:
GIT_DESCRIBE_TAG isn't set when the .git is a file
Sometimes the .git directory is actually a file.  This happens, for example, when you have a git submodule.  This causes this test to fail incorrectly: https://github.com/conda/conda-build/blob/master/conda_build/environ.py#L36

Unfortunately, the .git file might point to a real git repo in a relative URL, so just moving the directory to the conda build directory can break it being a git repo.



Discussion:

Resolving patch:
diff --git a/conda_build/environ.py b/conda_build/environ.py
--- a/conda_build/environ.py
+++ b/conda_build/environ.py
@@ -33,7 +33,7 @@ def get_git_build_info(src_dir):
     env = os.environ.copy()
     d = {}
     git_dir = join(src_dir, '.git')
-    if os.path.isdir(git_dir):
+    if os.path.exists(git_dir):
         env['GIT_DIR'] = git_dir
     else:
         return d


Test patch: