internlm/SWE-Fixer-Train-110K huggingface dataset:
Repository: conda/conda-build

Issue:
Regression regarding `[]` in jinja
Running into a regression when using the `[]` operator in jinja. This functionality was added by @stuarteberg in this PR ( https://github.com/conda/conda-build/pull/771 ). However, this appears to now be broken in `conda-build` 1.20.2. The current workaround would be to downgrade to `conda-build` 1.20.1, which does not have this problem. Here are some examples.

```
Error: Invalid selector in meta.yaml line 40:
    - python {{ environ['PY_VER'] + '*' }}  # [win]
```

See this [comment](https://github.com/conda-forge/hdf5-feedstock/pull/13#issuecomment-219302083) and PR for more details.

```
Error: Invalid selector in meta.yaml line 12:
  fn:  graphviz-{{ version[:-2] }}.zip  # [win]
```

Also see this [comment](https://github.com/conda-forge/staged-recipes/pull/568#issuecomment-219250321) for context.

cc @ccordoba12 @msarahan



Discussion:

Resolving patch:
diff --git a/conda_build/metadata.py b/conda_build/metadata.py
--- a/conda_build/metadata.py
+++ b/conda_build/metadata.py
@@ -69,7 +69,15 @@ def ns_cfg():
     return d
 
 
-sel_pat = re.compile(r'(.+?)\s*(#.*)?\[(.+)\](?(2).*)$')
+# Selectors must be either:
+# - at end of the line
+# - embedded (anywhere) within a comment
+#
+# Notes:
+# - [([^\[\]]+)\] means "find a pair of brackets containing any
+#                 NON-bracket chars, and capture the contents"
+# - (?(2).*)$ means "allow trailing characters iff group 2 (#.*) was found."
+sel_pat = re.compile(r'(.+?)\s*(#.*)?\[([^\[\]]+)\](?(2).*)$')
 
 
 def select_lines(data, namespace):


Test patch:
diff --git a/tests/test-recipes/metadata/jinja_vars/meta.yaml b/tests/test-recipes/metadata/jinja_vars/meta.yaml
--- a/tests/test-recipes/metadata/jinja_vars/meta.yaml
+++ b/tests/test-recipes/metadata/jinja_vars/meta.yaml
@@ -4,7 +4,7 @@ package:
 
 build:
   number: 123
-  string: {{CONDA_TEST_VAR}}_{{PKG_BUILDNUM}}
+  string: {{CONDA_TEST_VAR[:]}}_{{PKG_BUILDNUM}} # [True]
 
   script_env:
     - CONDA_TEST_VAR
diff --git a/tests/test_metadata.py b/tests/test_metadata.py
--- a/tests/test_metadata.py
+++ b/tests/test_metadata.py
@@ -9,26 +9,40 @@ def test_select_lines():
     lines = """
 test
 test [abc] no
+test [abc] # no
 
 test [abc]
 test # [abc]
 test # [abc] yes
 test # stuff [abc] yes
+test {{ JINJA_VAR[:2] }}
+test {{ JINJA_VAR[:2] }} # stuff [abc] yes
+test {{ JINJA_VAR[:2] }} # stuff yes [abc]
+test {{ JINJA_VAR[:2] }} # [abc] stuff yes
+{{ environ["test"] }}  # [abc]
 """
 
     assert select_lines(lines, {'abc': True}) == """
 test
 test [abc] no
+test [abc] # no
 
 test
 test
 test
 test
+test {{ JINJA_VAR[:2] }}
+test {{ JINJA_VAR[:2] }}
+test {{ JINJA_VAR[:2] }}
+test {{ JINJA_VAR[:2] }}
+{{ environ["test"] }}
 """
     assert select_lines(lines, {'abc': False}) == """
 test
 test [abc] no
+test [abc] # no
 
+test {{ JINJA_VAR[:2] }}
 """
Repository: conda/conda-build

Issue:
conda skeleton ignores the proxy environment variables
When an Enterprise client is operating from behind a proxy server, the conda skeleton command appears to ignore the availability of proxy settings in the $HTTP_PROXY, $HTTPS_PROXY and $NO_PROXY environment variables and fails connecting to pypi unless proxy settings are specified in the conda configuration file (observed with conda 3.16.0 on linux rhel6).

It would be really great if this behavior could be fixed and conda skeleton made use of the environment settings (as conda already does in most cases).



Discussion:
@JeremyEnns I'm working on this.   It looks to me like we need to be explicit in the version of requests that gets installed with conda.  I see there are changes in requests from version to version recently with respect to 
- whether proxy variables can be taken from the environment, 
- whether the request scheme has to be included in the proxy variable, and 
- whether NO_PROXY env variable is respected for domains excluded from the HTTP_PROXY.  

See [http://docs.python-requests.org/en/latest/community/updates/#id29](http://docs.python-requests.org/en/latest/community/updates/#id29)

In conda, there is an `install_requires` entry in `setup.py` for requests but we don't have a requirements file to specify which version. Would adding a recent requests version to a requirements file fix this, @csoja ?

Do you know @JeremyEnns whether this bug occurred when using Miniconda-based install or Anaconda.  My guess is Miniconda because Anaconda would have included the most recent requests version that does proxy environment variables.

I have version 2.9.1 of requests installed. I think the offending part is the function `get_xmlrpc_client` in `pypi.py`:

```
proxies = get_proxy_servers()

if proxies:
    transport = RequestsTransport()
else:
    transport = None
```

If I unconditionally use the `RequestsTransport` here, then it will use the proxy servers defined in the environment variables correctly.


Resolving patch:
diff --git a/conda_build/pypi.py b/conda_build/pypi.py
--- a/conda_build/pypi.py
+++ b/conda_build/pypi.py
@@ -22,7 +22,6 @@
 
 from conda.cli.common import spec_from_line
 from conda.compat import input, configparser, StringIO, string_types, PY3
-from conda.config import get_proxy_servers
 from conda.connection import CondaSession
 from conda.fetch import (download, handle_proxy_407)
 from conda.install import rm_rf
@@ -252,13 +251,7 @@ def _build_url(self, host, handler):
 
 
 def get_xmlrpc_client(pypi_url):
-    proxies = get_proxy_servers()
-
-    if proxies:
-        transport = RequestsTransport()
-    else:
-        transport = None
-    return ServerProxy(pypi_url, transport=transport)
+    return ServerProxy(pypi_url, transport=RequestsTransport())
 
 
 def main(args, parser):


Test patch:
Repository: conda/conda-build

Issue:
Keep origin recipe meta.yaml
@ilanschnell suggested that rather than renaming meta.yaml into meta.yaml.template and writing a static meta.yaml, it should instead be the original meta.yaml, with a static meta.yaml.rendered.



Discussion:

Resolving patch:
diff --git a/conda_build/build.py b/conda_build/build.py
--- a/conda_build/build.py
+++ b/conda_build/build.py
@@ -192,17 +192,16 @@ def create_info_files(m, files, include_recipe=True):
                 shutil.copytree(src_path, dst_path)
             else:
                 shutil.copy(src_path, dst_path)
-        os.rename(join(recipe_dir, "meta.yaml"), join(recipe_dir, "meta.yaml.template"))
-
-    # store the rendered meta.yaml file, plus information about where it came from
-    #    and what version of conda-build created it
-    metayaml = output_yaml(m)
-    with open(join(config.info_dir, "meta.yaml"), 'w') as f:
-        f.write("# This file created by conda-build {}\n".format(__version__))
-        f.write("# meta.yaml template originally from:\n")
-        f.write("# " + source.get_repository_info(m.path) + "\n")
-        f.write("# ------------------------------------------------\n\n")
-        f.write(metayaml)
+
+        # store the rendered meta.yaml file, plus information about where it came from
+        #    and what version of conda-build created it
+        metayaml = output_yaml(m)
+        with open(join(recipe_dir, "meta.yaml.rendered"), 'w') as f:
+            f.write("# This file created by conda-build {}\n".format(__version__))
+            f.write("# meta.yaml template originally from:\n")
+            f.write("# " + source.get_repository_info(m.path) + "\n")
+            f.write("# ------------------------------------------------\n\n")
+            f.write(metayaml)
 
     license_file = m.get_value('about/license_file')
     if license_file:


Test patch:
Repository: conda/conda-build

Issue:
Conda-build issue for an example in the tutorial for pyinstrument
((Anaconda3)) C:\Users\Rajesh>conda build pyinstrument
Removing old build environment
BUILD START: pyinstrument-0.10.1-py35_0
An unexpected error has occurred, please consider sending the
following traceback to the conda GitHub issue tracker at:

```
https://github.com/conda/conda-build/issues
```

Include the output of the command 'conda info' in your report.

Traceback (most recent call last):
  File "C:\Anaconda3\Scripts\conda-build-script.py", line 4, in <module>
    sys.exit(main())
  File "C:\Anaconda3\lib\site-packages\conda_build\main_build.py", line 144, in main
    args_func(args, p)
  File "C:\Anaconda3\lib\site-packages\conda_build\main_build.py", line 389, in args_func
    args.func(args, p)
  File "C:\Anaconda3\lib\site-packages\conda_build\main_build.py", line 332, in execute
    dirty=args.dirty, activate=args.activate)
  File "C:\Anaconda3\lib\site-packages\conda_build\build.py", line 478, in build
    has_vcs_available = os.path.isfile(external.find_executable(vcs_executable))
  File "C:\Anaconda3\lib\genericpath.py", line 30, in isfile
    st = os.stat(path)
TypeError: stat: can't specify None for path argument

((Anaconda3)) C:\Users\Rajesh>



Discussion:
I think that you are suffering from #1061 - basically, when the build process activates an environment, and an environment is already active, you can lose entries from PATH.  In your case, you are losing your PATH to your VCS tool.  That's how the output of find_executable is becoming None - there is no way for the vcs_executable to be None: https://github.com/conda/conda-build/blob/1.21.x/conda_build/build.py#L476-L478

There is a proposed fix at https://github.com/conda/conda/pull/2917 - until then, you have a few options:
1. Try to build without an environment active to begin with
2. list the appropriate version control system (VCS) in the build dependencies (git, svn, or whatever it uses)
3. call conda build with the --no-activate argument


Resolving patch:
diff --git a/conda_build/build.py b/conda_build/build.py
--- a/conda_build/build.py
+++ b/conda_build/build.py
@@ -475,7 +475,7 @@ def build(m, post=None, include_recipe=True, keep_old_work=False,
                 vcs_source = m.uses_vcs_in_build()
                 if vcs_source and vcs_source not in specs:
                     vcs_executable = "hg" if vcs_source == "mercurial" else vcs_source
-                    has_vcs_available = os.path.isfile(external.find_executable(vcs_executable))
+                    has_vcs_available = os.path.isfile(external.find_executable(vcs_executable) or "")
                     if not has_vcs_available:
                         if (vcs_source != "mercurial" or
                                 not any(spec.startswith('python') and "3." in spec


Test patch:
Repository: conda/conda-build

Issue:
meta.yaml selectors are not processed correctly
Selectors in `meta.yaml` work fine as long as the yaml lines containing them are not enclosed in single or double quotes:

``` yaml
 - pandas >=1.0   # [win]
```

However, "#" is a special character in yaml, which means that the entire value should be enclosed in quotes. As a result, yaml recipes generated with `yaml.dump()` look like this:

``` yaml
 - 'pandas >=1.0   # [win]'
 - "pandas >=1.0   # [win]"
```

When processing the selector, `conda build` strips out the selector itself (`# [win]`), but leaves the opening single/double quote:

``` yaml
 - 'pandas >=1.0
```

which is no longer a valid yaml file and `conda build` fails.



Discussion:
I think this is just a reflection of the fact that conda's defined syntax for `meta.yaml` is not, in fact, valid YAML. Therefore, we can't expect the raw output of `yaml.dump()` to necessarily valid input for the `meta.yaml` parser.

Perhaps, if they were to do it all over again, the conda designers would have named the file `meta.yaml.in` to reflect the fact that some preprocessing will be performed (selectors, jinja).

Anyway, if you're programmatically producing recipes, I think you'll need to come up with a workaround for this issue.  Something like:

``` python
yaml_text = yaml.dump({"requirements" : {"run" : ["pandas <HASH> [win]"]}}, default_flow_style=False)
print yaml_text.replace("<HASH>", "#")
```

``` yaml
requirements:
  run:
  - pandas # [win]
```

One simple solution would be to change the file extension to "notyaml".

We need to fix the parser. It's a trivial bug. I'll do some experiments and submit a PR.


Resolving patch:
diff --git a/conda_build/metadata.py b/conda_build/metadata.py
--- a/conda_build/metadata.py
+++ b/conda_build/metadata.py
@@ -89,8 +89,14 @@ def ns_cfg():
 
 def select_lines(data, namespace):
     lines = []
+
     for i, line in enumerate(data.splitlines()):
         line = line.rstrip()
+
+        trailing_quote = ""
+        if line and line[-1] in ("'", '"'):
+            trailing_quote = line[-1]
+
         if line.lstrip().startswith('#'):
             # Don't bother with comment only lines
             continue
@@ -99,7 +105,7 @@ def select_lines(data, namespace):
             cond = m.group(3)
             try:
                 if eval(cond, namespace, {}):
-                    lines.append(m.group(1))
+                    lines.append(m.group(1) + trailing_quote)
             except:
                 sys.exit('''\
 Error: Invalid selector in meta.yaml line %d:


Test patch:
diff --git a/tests/test_metadata.py b/tests/test_metadata.py
--- a/tests/test_metadata.py
+++ b/tests/test_metadata.py
@@ -12,8 +12,8 @@ def test_select_lines():
 test [abc] # no
 
 test [abc]
-test # [abc]
-test # [abc] yes
+ 'quoted # [abc] '
+ "quoted # [abc] yes "
 test # stuff [abc] yes
 test {{ JINJA_VAR[:2] }}
 test {{ JINJA_VAR[:2] }} # stuff [abc] yes
@@ -28,8 +28,8 @@ def test_select_lines():
 test [abc] # no
 
 test
-test
-test
+ 'quoted'
+ "quoted"
 test
 test {{ JINJA_VAR[:2] }}
 test {{ JINJA_VAR[:2] }}
Repository: conda/conda-build

Issue:
Build failure on appveyor that is new in 1.21.6
I tried out 1.21.6 tonight and got a build fail that I do not get with 1.21.5 (though the version of 1.21.5 I'm using is patched to take pin-numpy and setup-options).

A build that fails is at: https://ci.appveyor.com/project/Astropy/conda-channel-astropy/build/1.0.172#L1183

If you look at the history of the builds for that project, the one immediately before it, which uses 1.21.5, passes (I tried switching back and forth a couple times in case it was a fluke).



Discussion:
Yikes, something is trashing os.environ.  Looks like something that is fine for a single build, but bad for conda-build-all.  I have encountered things like this while playing with #1119, but had not seen it with 1.21.6.  I'll look into it.

I believe https://github.com/conda-forge/staged-recipes/pull/1111#issuecomment-234246070 is probably related to this, right?

Yeah, looks related.  Could be an issue in either conda-build or conda-build-all - but given that conda-build 1.21.5 works, perhaps more the former.

If you need a test case let me know and I can try it on https://github.com/conda-forge/humanfriendly-feedstock/pull/1

Must be something to do with [these changes](https://github.com/conda/conda-build/compare/1.21.5...1.21.6#diff-ac55d744935b1f37a3369258b88e8f1fR735) since they are the only thing that messes with the environ?

yes, @patricksnape that seems likely.  At the least, I should limit my env modifications to PATH.


Resolving patch:
diff --git a/conda_build/build.py b/conda_build/build.py
--- a/conda_build/build.py
+++ b/conda_build/build.py
@@ -732,12 +732,8 @@ def test(m, move_broken=True, activate=True):
             # not sure how this shakes out
             specs += ['lua %s*' % environ.get_lua_ver()]
 
-        # FIXME: stupid hack to put test prefix on PATH so that runtime libs can be found
-        old_env = os.environ.copy()
-        os.environ = prepend_bin_path(os.environ.copy(), config.test_prefix, True)
         create_env(config.test_prefix, specs)
-        os.environ = old_env
-        env = dict(os.environ)
+        env = dict(os.environ.copy())
         env.update(environ.get_dict(m, prefix=config.test_prefix))
 
         if not activate:


Test patch:
Repository: conda/conda-build

Issue:
conda build --output should not check out sources
It used not to, but at some point that changed, and now trying to figure where the built package is wastes a lot of time.



Discussion:
conda build should only be downloading source if it is necessary to render the recipe.  Do you use git variables in your jinja2 templates?  If so, you have to download the source before your recipe's build string can be determined.

No. This is the complete `meta.yaml` file, for the reference:

```
package:
  name: rustc
  version: "1.12.0"

source:
  git_url: https://github.com/m-labs/rust
  git_rev: artiq

build:
  number: 1

requirements:
  build:
    - system [linux]
    - cmake
    - llvm-or1k
    - curl
  run:
    - system [linux]
    - llvm-or1k

about:
  home: http://rust-lang.org/
  license: MIT/Apache
  summary: A safe, concurrent, practical language
```

Yes, that recipe should not be downloading source. I will look into it. 


Resolving patch:
diff --git a/conda_build/metadata.py b/conda_build/metadata.py
--- a/conda_build/metadata.py
+++ b/conda_build/metadata.py
@@ -728,19 +728,10 @@ def __repr__(self):
         return self.__str__()
 
     def uses_vcs_in_meta(self):
-        """returns true if recipe contains metadata associated with version control systems.
+        """returns name of vcs used if recipe contains metadata associated with version control systems.
         If this metadata is present, a download/copy will be forced in parse_or_try_download.
         """
         vcs_types = ["git", "svn", "hg"]
-        if "source" in self.meta:
-            for vcs in vcs_types:
-                if vcs + "_url" in self.meta["source"]:
-                    # translate command name to package name.
-                    # If more than hg, need a dict for this.
-                    if vcs == "hg":
-                        vcs = "mercurial"
-                    return vcs
-
         # We would get here if we use Jinja2 templating, but specify source with path.
         with open(self.meta_path) as f:
             metayaml = f.read()


Test patch:
Repository: conda/conda-build

Issue:
conda clean -a fails with conda build 2.0.0beta2
```
$ conda clean -a
Cache location: 
There are no tarballs to remove
Cache location: 
There are no unused packages to remove
An unexpected error has occurred, please consider sending the
following traceback to the conda GitHub issue tracker at:

    https://github.com/conda/conda/issues

Include the output of the command 'conda info' in your report.


Traceback (most recent call last):
  File "/home/eric/miniconda/bin/conda", line 6, in <module>
    sys.exit(conda.cli.main())
  File "/home/eric/miniconda/lib/python3.5/site-packages/conda/cli/main.py", line 120, in main
    exit_code = args_func(args, p)
  File "/home/eric/miniconda/lib/python3.5/site-packages/conda/cli/main.py", line 130, in args_func
    exit_code = args.func(args, p)
  File "/home/eric/miniconda/lib/python3.5/site-packages/conda/cli/main_clean.py", line 455, in execute
    json_result['source_cache'] = find_source_cache()
  File "/home/eric/miniconda/lib/python3.5/site-packages/conda/cli/main_clean.py", line 362, in find_source_cache
    'source cache': conda_build.source.SRC_CACHE,
AttributeError: module 'conda_build.source' has no attribute 'SRC_CACHE'
```

and the relevant bits from my conda environment

```
$ conda list -e | grep conda
Using Anaconda Cloud api site https://api.anaconda.org
# $ conda create --name <env> --file <this file>
anaconda-client=1.4.0=py35_0
conda=4.1.11=py35_1
conda-env=2.5.2=py35_0
conda-execute=0.6.0=py35_0
```

```
$ pip list | grep conda
anaconda-client (1.4.0)
conda (4.1.11)
conda-build (2.0.0b2, /home/eric/dev/conda/conda-build)
conda-build-all (0.13.3, /home/eric/dev/conda/conda-build-all)
conda-env (2.5.0a0)
conda-execute (0.6.0)
```



Discussion:
Thanks.  This requires a patch to conda.  I'll look into it.


Resolving patch:
diff --git a/conda_build/config.py b/conda_build/config.py
--- a/conda_build/config.py
+++ b/conda_build/config.py
@@ -330,5 +330,8 @@ def show(config):
     print('build packages directory:', config.bldpkgs_dir)
 
 
+# legacy exports for conda
+croot = Config().croot
+
 if __name__ == '__main__':
     show(Config())
diff --git a/conda_build/source.py b/conda_build/source.py
--- a/conda_build/source.py
+++ b/conda_build/source.py
@@ -15,6 +15,13 @@
 from conda_build.os_utils import external
 from conda_build.utils import tar_xf, unzip, safe_print_unicode, copy_into, on_win
 
+# legacy exports for conda
+from .config import Config as _Config
+SRC_CACHE = _Config().src_cache
+GIT_CACHE = _Config().git_cache
+SVN_CACHE = _Config().svn_cache
+HG_CACHE = _Config().hg_cache
+
 if on_win:
     from conda_build.utils import convert_unix_path_to_win
 


Test patch:
Repository: conda/conda-build

Issue:
__conda_version__ deprecation warning in conda build 2.0.0
Hi,

support for getting the post-build metadata from txt files is apparently still present in conda build 2.0.0, but the deprecation warnings still refer to this version ("Deprecation warning: support for **conda_version** will be removed in Conda build 2.0").

were these changes postponed? and if so, to which version? 

Thank you very much in advance,
riccardo



Discussion:
These changes have been postponed, probably until the next major version.  That's probably 1-2 months away.

Thanks for the clarification @msarahan 


Resolving patch:
diff --git a/conda_build/post.py b/conda_build/post.py
--- a/conda_build/post.py
+++ b/conda_build/post.py
@@ -433,26 +433,26 @@ def get_build_metadata(m, config):
     if "build" not in m.meta:
         m.meta["build"] = {}
     if exists(join(src_dir, '__conda_version__.txt')):
-        print("Deprecation warning: support for __conda_version__ will be removed in Conda build 2.0."  # noqa
+        print("Deprecation warning: support for __conda_version__ will be removed in Conda build 3.0."  # noqa
               "Try Jinja templates instead: "
-              "http://conda.pydata.org/docs/building/environment-vars.html#git-environment-variables")  # noqa
+              "http://conda.pydata.org/docs/building/meta-yaml.html#templating-with-jinja")
         with open(join(src_dir, '__conda_version__.txt')) as f:
             version = f.read().strip()
             print("Setting version from __conda_version__.txt: %s" % version)
             m.meta['package']['version'] = version
     if exists(join(src_dir, '__conda_buildnum__.txt')):
-        print("Deprecation warning: support for __conda_buildnum__ will be removed in Conda build 2.0."  # noqa
+        print("Deprecation warning: support for __conda_buildnum__ will be removed in Conda build 3.0."  # noqa
               "Try Jinja templates instead: "
-              "http://conda.pydata.org/docs/building/environment-vars.html#git-environment-variables")  # noqa
+              "http://conda.pydata.org/docs/building/meta-yaml.html#templating-with-jinja")
         with open(join(src_dir, '__conda_buildnum__.txt')) as f:
             build_number = f.read().strip()
             print("Setting build number from __conda_buildnum__.txt: %s" %
                   build_number)
             m.meta['build']['number'] = build_number
     if exists(join(src_dir, '__conda_buildstr__.txt')):
-        print("Deprecation warning: support for __conda_buildstr__ will be removed in Conda build 2.0."  # noqa
+        print("Deprecation warning: support for __conda_buildstr__ will be removed in Conda build 3.0."  # noqa
               "Try Jinja templates instead: "
-              "http://conda.pydata.org/docs/building/environment-vars.html#git-environment-variables")  # noqa
+              "http://conda.pydata.org/docs/building/meta-yaml.html#templating-with-jinja")
         with open(join(src_dir, '__conda_buildstr__.txt')) as f:
             buildstr = f.read().strip()
             print("Setting version from __conda_buildstr__.txt: %s" % buildstr)


Test patch: