internlm/SWE-Fixer-Train-110K huggingface dataset:
Repository: conda/conda-build

Issue:
Conda multi output recipe takes 9+ minutes to render
I am using conda build 3.13.0, but the behaviour is similar in the recent versions.
```
/t/llvm-compilers-linux ❯❯❯ /usr/bin/time ~/m3/bin/conda render . > /dev/null                                                                                                                               ✘ 127 
645.78user 0.71system 10:49.03elapsed 99%CPU (0avgtext+0avgdata 88532maxresident)k
96inputs+32outputs (1major+55066minor)pagefaults 0swaps
```
CB takes 645 seconds just to render this [recipe](https://github.com/conda/conda-build/files/2347465/llvm-compilers-linux.zip)



Discussion:
If I run strace on it to see how many times it opens meta.yaml, the count reaches to 2973!

Resolving patch:
diff --git a/conda_build/metadata.py b/conda_build/metadata.py
--- a/conda_build/metadata.py
+++ b/conda_build/metadata.py
@@ -21,6 +21,7 @@
 from .conda_interface import string_types
 
 from conda_build import exceptions, utils, variants, environ
+from conda_build.conda_interface import memoized
 from conda_build.features import feature_list
 from conda_build.config import Config, get_or_merge_config
 from conda_build.utils import (ensure_list, find_recipe, expand_globs, get_installed_packages,
@@ -745,6 +746,7 @@ def _filter_recipe_text(text, extract_pattern=None):
     return text
 
 
+@memoized
 def read_meta_file(meta_path):
     with open(meta_path, 'rb') as f:
         recipe_text = UnicodeDammit(f.read()).unicode_markup


Test patch:
Repository: conda/conda-build

Issue:
Can't get conda index to work with channel_name
From the updated docs on 3.14.0, `channel_name` is now a positional argument. However, when I do
```bash
$ conda index <build_dir> <channel_name>
```
I get
```
(...)
FileNotFoundError: [Errno 2] No such file or directory: '<path_prefix>/<channel_name>'
```
instead of a custom channel name in the outputted html files. 

Am I forgetting some special `argparse` magic or did #3091 introduce a bug?

I'm on conda 4.5.11 and conda-build 3.14.0


Discussion:
This should have been a flag, not a positional argument.  As a positional arg, it conflicts with the variable length input for the build dirs.  I will fix it.

Resolving patch:
diff --git a/conda_build/cli/main_index.py b/conda_build/cli/main_index.py
--- a/conda_build/cli/main_index.py
+++ b/conda_build/cli/main_index.py
@@ -30,10 +30,8 @@ def parse_args(args):
         package's metadata needs to be updated.""",
     )
     p.add_argument(
-        'channel_name',
-        help='Adding a channel name will create an index.html file within the subdir.',
-        nargs='?',
-        default=None,
+        "-n", "--channel-name",
+        help="Customize the channel name listed in each channel's index.html.",
     )
     p.add_argument(
         '-s', '--subdir',


Test patch:
Repository: conda/conda-build

Issue:
Indexing does not remove files that are no longer present
The repodata caching seems to keep files even when they no longer exist on disk.  These should be removed, even if they don't exist in the removals key.
Indexing does not remove files that are no longer present
The repodata caching seems to keep files even when they no longer exist on disk.  These should be removed, even if they don't exist in the removals key.


Discussion:

Resolving patch:
diff --git a/conda_build/index.py b/conda_build/index.py
--- a/conda_build/index.py
+++ b/conda_build/index.py
@@ -947,6 +947,7 @@ def index_subdir(self, subdir, verbose=False, progress=False):
             # calculate all the paths and figure out what we're going to do with them
             # add_set: filenames that aren't in the current/old repodata, but exist in the subdir
             add_set = fns_in_subdir - old_repodata_fns
+            remove_set = old_repodata_fns - fns_in_subdir
             update_set = self._calculate_update_set(
                 subdir, fns_in_subdir, old_repodata_fns, stat_cache, verbose=verbose, progress=progress
             )
@@ -954,7 +955,7 @@ def index_subdir(self, subdir, verbose=False, progress=False):
             #     and whose contents have changed based on file size or mtime. We're
             #     not using md5 here because it takes too long. If needing to do full md5 checks,
             #     use the --deep-integrity-check flag / self.deep_integrity_check option.
-            unchanged_set = sorted(old_repodata_fns - update_set)
+            unchanged_set = sorted(old_repodata_fns - update_set - remove_set)
             # unchanged_set: packages in old repodata whose information can carry straight
             #     across to new repodata
 


Test patch:
Repository: conda/conda-build

Issue:
The return of conda-skeleton-PyPI tussle 
Using:
```
conda-build 3.15.1
```
Triggered by:
```
conda skeleton pypi path.py --version 11.5.0
```
```
Traceback (most recent call last):
  File "/opt/conda/bin/conda-skeleton", line 11, in <module>
    sys.exit(main())
  File "/opt/conda/lib/python3.6/site-packages/conda_build/cli/main_skeleton.py", line 65, in main
    return execute(sys.argv[1:])
  File "/opt/conda/lib/python3.6/site-packages/conda_build/cli/main_skeleton.py", line 61, in execute
    version=args.version, config=config)
  File "/opt/conda/lib/python3.6/site-packages/conda_build/api.py", line 282, in skeletonize
    recursive=recursive, config=config, **kwargs)
  File "/opt/conda/lib/python3.6/site-packages/conda_build/skeletons/pypi.py", line 290, in skeletonize
    setup_options=setup_options)
  File "/opt/conda/lib/python3.6/site-packages/conda_build/skeletons/pypi.py", line 763, in get_package_metadata
    dep, marker = parse_dep_with_env_marker(dep)
  File "/opt/conda/lib/python3.6/site-packages/conda_build/skeletons/pypi.py", line 678, in parse_dep_with_env_marker
    match.group("env_mark_constraint"))
  File "/opt/conda/lib/python3.6/site-packages/conda_build/skeletons/pypi.py", line 666, in env_mark_lookup
    env_mark_table[env_mark_name]['constraint_trans_fn'](env_mark_constraint)))
  File "/opt/conda/lib/python3.6/site-packages/conda_build/skeletons/pypi.py", line 652, in _translate_python_constraint
    operator, value = constraint.split()
ValueError: not enough values to unpack (expected 2, got 1)
```


Discussion:

Resolving patch:
diff --git a/conda_build/skeletons/pypi.py b/conda_build/skeletons/pypi.py
--- a/conda_build/skeletons/pypi.py
+++ b/conda_build/skeletons/pypi.py
@@ -648,9 +648,13 @@ def convert_version(version):
 
 
 def _translate_python_constraint(constraint):
-    operator, value = constraint.split()
-    value = "".join(value.strip("'").strip('"').split(".")[:2])
-    return " ".join((operator, value))
+    parts = constraint.split()
+    translation = constraint
+    if len(parts) == 2:
+        operator, value = parts
+        value = "".join(value.strip("'").strip('"').split(".")[:2])
+        translation = " ".join((operator, value))
+    return translation
 
 
 def env_mark_lookup(env_mark_name, env_mark_constraint):


Test patch:
Repository: conda/conda-build

Issue:
allow overriding the .so check for noarch: python?
`compas` is a pure-python package that could be made `noarch: python` except that it [ships some `.so` and `.dll` files](https://github.com/compas-dev/compas/tree/master/src/compas/numerical/fd/__fd_cpp) (that it doesn't compile, so they're the same across platforms). That's triggering [this check](https://github.com/conda/conda-build/blob/19f5d7847ee8a4c5b979680595103fa6cc21e4b1/conda_build/noarch_python.py#L60-L66) and killing the build ([PR](https://github.com/conda-forge/compas-feedstock/pull/6#issuecomment-429024394), [failed build](https://circleci.com/gh/conda-forge/compas-feedstock/37?utm_campaign=vcs-integration-link&utm_medium=referral&utm_source=github-build-link)).

The check definitely makes sense in general. But maybe there should be a way to override it for cases like this?


Discussion:
Sure, seems fine to me.  Please consider submitting a PR.
How should this be handled? Remove this check and let --verify handle it, or should we just provide a flag to override it?
the right way in my mind is to remove the check in conda-build, and leave it in conda-verify.  Then let people see the warning, or use the ``--strict-verify`` for conda-build to error out when conda-verify has any unhappy result.
Sounds good, I'll submit a PR to remove the check.

Resolving patch:
diff --git a/conda_build/noarch_python.py b/conda_build/noarch_python.py
--- a/conda_build/noarch_python.py
+++ b/conda_build/noarch_python.py
@@ -57,13 +57,10 @@ def handle_file(f, d, prefix):
     if f.endswith(('.egg-info', '.pyc', '.pyo')):
         os.unlink(path)
 
-    # The presence of .so indicated this is not a noarch package
-    elif f.endswith(('.so', '.dll', '.pyd', '.exe', '.dylib')):
-        if f.endswith('.exe') and (isfile(os.path.join(prefix, f[:-4] + '-script.py')) or
-                                   basename(f[:-4]) in d['python-scripts']):
-            os.unlink(path)  # this is an entry point with a matching xx-script.py
-            return
-        _error_exit("Error: Binary library or executable found: %s" % f)
+    if f.endswith('.exe') and (isfile(os.path.join(prefix, f[:-4] + '-script.py')) or
+                               basename(f[:-4]) in d['python-scripts']):
+        os.unlink(path)  # this is an entry point with a matching xx-script.py
+        return
 
     elif 'site-packages' in f:
         nsp = join(prefix, 'site-packages')


Test patch:
Repository: conda/conda-build

Issue:
Test commands not running on Windows
This is a side effect of https://github.com/conda/conda-build/pull/3236

A very small reproducer is:
```
(base) C:\Users\Wani\Downloads\scratch>type meta.yaml
package:
   name: yes_sir
   version: 0.0.1

test:
   commands:
     - echo "Hello"
```

For the time being, downgrade conda-build to 3.16.1


Discussion:
Thanks for reporting @nehaljwani - this is affecting us also.
@msarahan - commit https://github.com/conda/conda-build/commit/a0de016ccf0e9d223b8da5ad12d71ec80a8fcae7 looks a bit suspicious

In particular, the change from using `test_file` to `shell_file`:

![image](https://user-images.githubusercontent.com/881019/48105004-72d4b000-e281-11e8-956e-0985838131fc.png)


That might well be a red-herring though as I'm not sure I follow the logic and haven't properly tried to debug it.



Resolving patch:
diff --git a/conda_build/build.py b/conda_build/build.py
--- a/conda_build/build.py
+++ b/conda_build/build.py
@@ -2110,7 +2110,7 @@ def test(recipedir_or_package_or_metadata, config, stats, move_broken=True):
         if shell_files:
             for shell_file in shell_files:
                 if utils.on_win:
-                    if os.path.splitext(shell_file) == ".bat":
+                    if os.path.splitext(shell_file)[1].lower() == ".bat":
                         tf.write('call "{test_file}"\n'.format(test_file=shell_file))
                         tf.write("IF %ERRORLEVEL% NEQ 0 exit 1\n")
                     else:


Test patch:
Repository: conda/conda-build

Issue:
conda-build 3.6.3 raises UnicodeDecodeError in libnetcdf feedstock
Using v3.6.3 fails with UnicodeDecodeError after the test step on Travis for the libnetcdf feedstock:
see logs at https://github.com/conda-forge/libnetcdf-feedstock/pull/62
Pinning it to 3.6.2 make the error disappear.



Discussion:
Pasting error from linked issue here:

```
Traceback (most recent call last):
  File "/Users/travis/miniconda3/lib/python3.6/threading.py", line 916, in _bootstrap_inner
    self.run()
  File "/Users/travis/miniconda3/lib/python3.6/threading.py", line 864, in run
    self._target(*self._args, **self._kwargs)
  File "/Users/travis/miniconda3/lib/python3.6/site-packages/conda_build/utils.py", line 197, in rewriter
    line = r.readline()
  File "/Users/travis/miniconda3/lib/python3.6/codecs.py", line 321, in decode
    (result, consumed) = self._buffer_decode(data, self.errors, final)
UnicodeDecodeError: 'utf-8' codec can't decode byte 0xff in position 201: invalid start byte
```

This error points to problems with the output rewriting introduced in #3258.  We should probably try to rewrite, but if problems occur, fall back to the old unaltered output. 

Resolving patch:
diff --git a/conda_build/utils.py b/conda_build/utils.py
--- a/conda_build/utils.py
+++ b/conda_build/utils.py
@@ -195,15 +195,22 @@ def _setup_rewrite_pipe(env):
 
     def rewriter():
         while True:
-            line = r.readline()
-            if not line:
-                # reading done
-                r.close()
-                os.close(w_fd)
-                return
-            for s, key in replacements.items():
-                line = line.replace(s, replacement_t.format(key))
-            sys.stdout.write(line)
+            try:
+                line = r.readline()
+                if not line:
+                    # reading done
+                    r.close()
+                    os.close(w_fd)
+                    return
+                for s, key in replacements.items():
+                    line = line.replace(s, replacement_t.format(key))
+                sys.stdout.write(line)
+            except UnicodeDecodeError:
+                try:
+                    txt = os.read(r, 10000)
+                    sys.stdout.write(txt or '')
+                except TypeError:
+                    pass
 
     t = Thread(target=rewriter)
     t.daemon = True


Test patch:
Repository: conda/conda-build

Issue:
Fix for PIP_NO_DEPENDENCIES was not made to windows.py
PR #3271 fixed #3254 by inverting the value of the PIP_NO_DEPENDENCIES flag, but the fix seems to be incomplete, as it did not change the value in the `windows.py` script. Compare:

https://github.com/conda/conda-build/blob/c8eae4960242dd425f2f0eb15cfe19488497cfac/conda_build/windows.py#L308

https://github.com/conda/conda-build/blob/c8eae4960242dd425f2f0eb15cfe19488497cfac/conda_build/build.py#L1862

It's a simple fix, but I'm not in a position to test it (don't have a Windows machine), so I won't issue the PR.



Discussion:

Resolving patch:
diff --git a/conda_build/build.py b/conda_build/build.py
--- a/conda_build/build.py
+++ b/conda_build/build.py
@@ -1856,7 +1856,7 @@ def write_build_scripts(m, script, build_file):
     # Note that pip env "NO" variables are inverted logic.
     #      PIP_NO_BUILD_ISOLATION=False means don't use build isolation.
     #
-    env["PIP_NO_BUILD_ISOLATION"] = False
+    env["PIP_NO_BUILD_ISOLATION"] = 'False'
     # some other env vars to have pip ignore dependencies.
     # we supply them ourselves instead.
     env["PIP_NO_DEPENDENCIES"] = True
@@ -1876,7 +1876,7 @@ def write_build_scripts(m, script, build_file):
     env_file = join(m.config.work_dir, 'build_env_setup.sh')
     with open(env_file, 'w') as bf:
         for k, v in env.items():
-            if v:
+            if v != '' and v is not None:
                 bf.write('export {0}="{1}"\n'.format(k, v))
 
         if m.activate_build_script:
diff --git a/conda_build/windows.py b/conda_build/windows.py
--- a/conda_build/windows.py
+++ b/conda_build/windows.py
@@ -264,7 +264,7 @@ def write_build_scripts(m, env, bld_bat):
         # more debuggable with echo on
         fo.write('@echo on\n')
         for key, value in env.items():
-            if value:
+            if value != '' and value is not None:
                 fo.write('set "{key}={value}"\n'.format(key=key, value=value))
         if not m.uses_new_style_compiler_activation:
             fo.write(msvc_env_cmd(bits=m.config.host_arch, config=m.config,
@@ -301,11 +301,11 @@ def build(m, bld_bat, stats, provision_only=False):
     # Note that pip env "NO" variables are inverted logic.
     #      PIP_NO_BUILD_ISOLATION=False means don't use build isolation.
     #
-    env["PIP_NO_BUILD_ISOLATION"] = False
+    env["PIP_NO_BUILD_ISOLATION"] = 'False'
     # some other env vars to have pip ignore dependencies.
     # we supply them ourselves instead.
     #    See note above about inverted logic on "NO" variables
-    env["PIP_NO_DEPENDENCIES"] = False
+    env["PIP_NO_DEPENDENCIES"] = True
     env["PIP_IGNORE_INSTALLED"] = True
 
     # pip's cache directory (PIP_NO_CACHE_DIR) should not be


Test patch:
Repository: conda/conda-build

Issue:
conda build documentation needs update 
The document here needs update I think... 
https://github.com/conda/conda-build/blob/master/docs/source/resources/commands/conda-build.rst

Specifically the `--test` part

> 
       -t, --test
              Test package (assumes package  is  already  built).   RECIPE_DIR
              argument  can  be  either recipe directory, in which case source
              download may be necessary to resolve package version, or path to
              built  package  .tar.bz2 file, in which case no source is neces-
              sary.

This is no longer the case anymore right? As in you can no longer run `conda build --test --someOtherOptions conda.recipe`, because: https://github.com/conda/conda-build/blob/077880c00bf22f0b3cde28e03c977efac0fa9846/conda_build/build.py#L1667

It would be much appreciated if there are some examples to the links as in how do we test conda-build packages with `conda build -V` >= 3.16.0 ?

Is this the new way to test conda-build packages?
https://github.com/conda/conda-build#testing  



Discussion:

Resolving patch:
diff --git a/conda_build/cli/main_build.py b/conda_build/cli/main_build.py
--- a/conda_build/cli/main_build.py
+++ b/conda_build/cli/main_build.py
@@ -73,9 +73,8 @@ def parse_args(args):
     p.add_argument(
         '-t', "--test",
         action="store_true",
-        help="Test package (assumes package is already built).  RECIPE_DIR argument can be either "
-        "recipe directory, in which case source download may be necessary to resolve package "
-        "version, or path to built package .tar.bz2 file, in which case no source is necessary.",
+        help="Test package (assumes package is already built).  RECIPE_DIR argument must be a "
+        "path to built package .tar.bz2 file.",
     )
     p.add_argument(
         '--no-test',


Test patch:
Repository: conda/conda-build

Issue:
test/source_files recursive glob doesn't work
The following recipe snippet doesn't work as I expect:
```yaml
test:
  source_files:
    - tests/**/*.py
```
Namely, it doesn't find the files of the form `tests/test_something.py`, but does find files similar to `tests/subdir/test_somethingelse.py`.

This is because the code in question, https://github.com/conda/conda-build/blob/63b1f0b98cfdff96e389e5015ba72b67ba06cc22/conda_build/build.py#L396
uses the stdlib glob (without recursion):
https://github.com/conda/conda-build/blob/63b1f0b98cfdff96e389e5015ba72b67ba06cc22/conda_build/build.py#L8
There's a PR that introduced glob2 (#2146) but that somehow overlooked the above line.

I myself am bit hesitant to write a PR that simply changes the above import to glob2 since it also touches other glob statements in `build.py`. Hence this issue.

As a side note, there's a generic `glob` definition in `conda-build/conda_build/utils.py` that turns recursion on by default. But using that won't work for Python < 3.5 since glob recursion was introduced in the stdlib in 3.5.


Discussion:
The function in utils.py should fall back to glob2 on python 2, I believe.  https://github.com/conda/conda-build/blob/master/conda_build/utils.py#L51-L75

The right fix here is to change build.py to import the glob function from utils.py

Is python <3.5 even relevant anymore?  I don't think so.  Everything older than 3.5 is already EOL, and Anaconda and Conda-forge don't even build packages for 3.5 anymore.  https://devguide.python.org/#status-of-python-branches

Resolving patch:
diff --git a/conda_build/build.py b/conda_build/build.py
--- a/conda_build/build.py
+++ b/conda_build/build.py
@@ -5,7 +5,6 @@
 
 from collections import deque, OrderedDict
 import fnmatch
-from glob import glob
 import io
 import json
 import os
@@ -56,7 +55,7 @@
 from .conda_interface import NoPackagesFoundError
 from .conda_interface import CondaError
 from .conda_interface import pkgs_dirs
-from .utils import env_var, tmp_chdir
+from .utils import env_var, glob, tmp_chdir
 
 from conda_build import __version__
 from conda_build import environ, source, tarcheck, utils


Test patch:
Repository: conda/conda-build

Issue:
has_prefix_files in windows with \\ path delimiter for relocation
I am having issues with a path that is coded in a python script under windows. In mac/linux conda-build finds the path and makes it relocatable, but not on windows. I think it may be that there are double \\ for path delimiter

### Actual Behavior

Path is hard-coded in conda package on windows (not on mac/linux). Build says that no hard coded path was found in the file, depite that there is. "Did not detect hard-coded path in Lib/site-packages/jcc/config.py from has_prefix_files"

One example text in that file is:
INCLUDES=['C:\\Users\\XXX\\AppData\\Local\\Continuum\\Anaconda3-430\\Library\\include', 'C:\\Users\\XXX\\AppData\\Local\\Continuum\\Anaconda3-430\\Library\\include\\win32']


### Expected Behavior

The hard-coded path should be replaced with the relocatable string marker.

### Steps to Reproduce


##### `Output of conda info`
```
          conda version : 4.5.12
    conda-build version : 3.17.6
         python version : 3.6.2.final.0


```




Discussion:
@petrushy I think I hit the same issue you ran into. Not sure if you have figured out any workaround.

I haven't confirmed it yet but I suspected it to do with this logic:
https://github.com/conda/conda-build/blob/3.18.9/conda_build/build.py#L246-L256

Resolving patch:
diff --git a/conda_build/build.py b/conda_build/build.py
--- a/conda_build/build.py
+++ b/conda_build/build.py
@@ -199,7 +199,11 @@ def have_prefix_files(files, prefix):
             except subprocess.CalledProcessError:
                 continue
         # HACK: this is basically os.path.relpath, just simpler and faster
-        rg_matches = [rg_match[prefix_len:] for rg_match in rg_matches]
+        # NOTE: path normalization needs to be in sync with create_info_files
+        if utils.on_win:
+            rg_matches = [rg_match.replace('\\', '/')[prefix_len:] for rg_match in rg_matches]
+        else:
+            rg_matches = [rg_match[prefix_len:] for rg_match in rg_matches]
     else:
         print("WARNING: Detecting which files contain PREFIX is slow, installing ripgrep makes it faster."
               " 'conda install ripgrep'")


Test patch:
Repository: conda/conda-build

Issue:
BUG warn on dangling symlinks
<!---
Thanks for opening a PR on conda-build!

Please include a news entry with your PR to help keep our changelog up to date!
There are instructions available at: https://regro.github.io/rever-docs/news.html

If there is specific issue / feature request that this PR is addressing,
please link to the corresponding issue by using the `#issuenumber` syntax.

Thanks again!
-->

Closes #3844 

@isuruf for viz


Discussion:

Resolving patch:
diff --git a/conda_build/build.py b/conda_build/build.py
--- a/conda_build/build.py
+++ b/conda_build/build.py
@@ -8,6 +8,7 @@
 import io
 import json
 import os
+import warnings
 from os.path import isdir, isfile, islink, join, dirname
 import random
 import re
@@ -1280,6 +1281,7 @@ def build_info_files_json_v1(m, prefix, files, files_with_prefix):
         elif not isfile(path):
             # this is a softlink that points to nowhere, so is zero bytes
             file_info["size_in_bytes"] = 0
+            warnings.warn('file %s is a symlink with no target' % path, UserWarning)
         else:
             # softlink that points somewhere
             file_info["size_in_bytes"] = os.stat(path).st_size


Test patch: