internlm/SWE-Fixer-Train-110K huggingface dataset:
Repository: conda-forge/conda-smithy

Issue:
Not compatible with ruamel.yaml 0.16
Fails with,

```
Traceback (most recent call last):

  File "/home/travis/miniconda/bin/conda-smithy", line 10, in <module>

    sys.exit(main())

  File "/home/travis/miniconda/lib/python3.7/site-packages/conda_smithy/cli.py", line 470, in main

    args.subcommand_func(args)

  File "/home/travis/miniconda/lib/python3.7/site-packages/conda_smithy/cli.py", line 217, in __call__

    args.feedstock_directory, owner, repo

  File "/home/travis/miniconda/lib/python3.7/site-packages/conda_smithy/ci_register.py", line 351, in travis_token_update_conda_forge_config

    ] = travis_encrypt_binstar_token(slug, item)

  File "/home/travis/miniconda/lib/python3.7/contextlib.py", line 119, in __exit__

    next(self.gen)

  File "/home/travis/miniconda/lib/python3.7/site-packages/conda_smithy/utils.py", line 92, in update_conda_forge_config

    fh.write(yaml.dump(code))

  File "/home/travis/miniconda/lib/python3.7/site-packages/ruamel/yaml/main.py", line 448, in dump

    raise TypeError('Need a stream argument when not dumping from context manager')

TypeError: Need a stream argument when not dumping from context manager
```

cc @ocefpaf, @scopatz


Discussion:
@conda-forge/core 

Resolving patch:
diff --git a/conda_smithy/utils.py b/conda_smithy/utils.py
--- a/conda_smithy/utils.py
+++ b/conda_smithy/utils.py
@@ -88,5 +88,4 @@ def update_conda_forge_config(feedstock_directory):
 
     yield code
 
-    with open(forge_yaml, "w") as fh:
-        fh.write(yaml.dump(code))
+    yaml.dump(code, forge_yaml)


Test patch:
Repository: conda-forge/conda-smithy

Issue:
Drone sync
cc @mariusvniekerk, @conda-forge/arm-arch 

Needs to do `POST /api/user/repos?async=true` . This takes 100 seconds, so we don't want to do this for each feedstock in staged-recipes. A flag to skip this would be great and staged-recipes can pass that flag for the subsequent calls


Discussion:

Resolving patch:
diff --git a/conda_smithy/ci_register.py b/conda_smithy/ci_register.py
--- a/conda_smithy/ci_register.py
+++ b/conda_smithy/ci_register.py
@@ -139,14 +139,31 @@ def add_token_to_drone(user, project):
         "data": anaconda_token,
         "pull_request": False,
     })
+    if response.status_code != 200:
+        # Check that the token is in secrets already
+        session = drone_session()
+        response2 = session.get(f'/api/repos/{user}/{project}/secrets')
+        response2.raise_for_status()
+        for secret in response2.json():
+            if "BINSTAR_TOKEN" == secret["name"]:
+                return
+    response.raise_for_status()
+
+
+def drone_sync():
+    session = drone_session()
+    response = session.post('/api/user/repos?async=true')
     response.raise_for_status()
-    print(response.json())
 
 
 def add_project_to_drone(user, project):
     session = drone_session()
     response = session.post(f'/api/repos/{user}/{project}')
-    response.raise_for_status()
+    if response.status_code != 200:
+        # Check that the project is registered already
+        session = drone_session()
+        response = session.get(f'/api/repos/{user}/{project}')
+        response.raise_for_status()
 
 
 def regenerate_drone_webhooks(user, project):


Test patch:
Repository: conda-forge/conda-smithy

Issue:
refactor needed_families to top level
This way downstream tools can keep track of which license families need files.
<!--
Thank you for pull request.
Below are a few things we ask you kindly to self-check before getting a review. Remove checks that are not relevant.
-->
Checklist
* [ ] Added a ``news`` entry

<!--
Please note any issues this fixes using [closing keywords]( https://help.github.com/articles/closing-issues-using-keywords/ ):
-->

<!--
Please add any other relevant info below:
-->


Discussion:
Rebase plz
🤦‍♂ why did I merge that black one first!

Resolving patch:
diff --git a/conda_smithy/lint_recipe.py b/conda_smithy/lint_recipe.py
--- a/conda_smithy/lint_recipe.py
+++ b/conda_smithy/lint_recipe.py
@@ -46,6 +46,9 @@
 TEST_KEYS = {"imports", "commands"}
 TEST_FILES = ["run_test.py", "run_test.sh", "run_test.bat", "run_test.pl"]
 
+
+NEEDED_FAMILIES = ["gpl", "bsd", "mit", "apache", "psf"]
+
 sel_pat = re.compile(r"(.+?)\s*(#.*)?\[([^\[\]]+)\](?(2).*)$")
 jinja_pat = re.compile(r"\s*\{%\s*(set)\s+[^\s]+\s*=\s*[^\s]+\s*%\}")
 
@@ -278,9 +281,8 @@ def lintify(meta, recipe_dir=None, conda_forge=False):
     # 12a: License family must be valid (conda-build checks for that)
     license_family = about_section.get("license_family", license).lower()
     license_file = about_section.get("license_file", "")
-    needed_families = ["gpl", "bsd", "mit", "apache", "psf"]
     if license_file == "" and any(
-        f for f in needed_families if f in license_family
+        f for f in NEEDED_FAMILIES if f in license_family
     ):
         lints.append("license_file entry is missing, but is required.")
 


Test patch:
Repository: conda-forge/conda-smithy

Issue:
Linter crashes under certain conditions
<!--
Thanks for reporting your issue.
Please fill out the sections below.
-->
Issue:

The linter (version 3.6.8) crashes with:

```
File lint_recipe.py, line 485, in lintify
constraint = req.split(" ", 1)[1]
IndexError: list index out of range
```

This happens when non noarch build is triggered and the run section contains packages that start with `python` (e.g. python-dateutil) that do not have version contraints. 

Minimal example `meta.yaml`:
```
{% set name = "testapp" %}
{% set version = "0.0.1" %}

package:
  name: {{ name|lower }}
  version: {{ version }}

source:
  - url: https://github.com/apache/{{ name }}/archive/{{ version }}.tar.gz
    sha256: 194n4140184140194124914u1n0914j1041j04104014m1401420411244210410
  - url: https://raw.githubusercontent.com/apache/{{ name }}/master/LICENSE.txt
    sha256: 832u5y3285w89y537w05w49572j5nnm151591515213532mnn141249000090914

build:
  # noarch not set
  number: 0
  script:
    - "{{ PYTHON }} -m pip install . -vv"
  skip: True  # [win or py2k]
  entry_points:
    - superset = superset.bin.superset:cli

requirements:
  build:
    - nodejs
  host:
    - python
    - pip
  run:
    - python-dateutil # without version
```



Discussion:

Resolving patch:
diff --git a/conda_smithy/lint_recipe.py b/conda_smithy/lint_recipe.py
--- a/conda_smithy/lint_recipe.py
+++ b/conda_smithy/lint_recipe.py
@@ -462,12 +462,12 @@ def lintify(meta, recipe_dir=None, conda_forge=False):
             filtered_host_reqs = [
                 req
                 for req in host_reqs
-                if req == str(language) or req.startswith(str(language))
+                if req.partition(" ")[0] == str(language)
             ]
             filtered_run_reqs = [
                 req
                 for req in run_reqs
-                if req == str(language) or req.startswith(str(language))
+                if req.partition(" ")[0] == str(language)
             ]
             if filtered_host_reqs and not filtered_run_reqs:
                 lints.append(


Test patch:
diff --git a/tests/test_lint_recipe.py b/tests/test_lint_recipe.py
--- a/tests/test_lint_recipe.py
+++ b/tests/test_lint_recipe.py
@@ -918,6 +918,12 @@ def test_python_requirements(self):
             lints,
         )
 
+        meta = {
+            "requirements": {"host": ["python"], "run": ["python-dateutil"]}
+        }
+        # Test that this doesn't crash
+        lints, hints = linter.lintify(meta)
+
     def test_r_base_requirements(self):
         meta = {"requirements": {"host": ["r-base >=3.5"]}}
         lints, hints = linter.lintify(meta)
Repository: conda-forge/conda-smithy

Issue:
deprecate the binstar token in the conda-forge.yml file
Travis now has tokens stored directly. We should remove any use of the encrypted token in the file.


Discussion:

Resolving patch:
diff --git a/conda_smithy/ci_register.py b/conda_smithy/ci_register.py
--- a/conda_smithy/ci_register.py
+++ b/conda_smithy/ci_register.py
@@ -488,6 +488,28 @@ def travis_configure(user, project):
             response.raise_for_status()
 
 
+def add_token_to_travis(user, project):
+    """Add the BINSTAR_TOKEN to travis."""
+
+    headers = travis_headers()
+
+    repo_info = travis_get_repo_info(user, project)
+    repo_id = repo_info["id"]
+
+    data = {
+        "env_var.name": "BINSTAR_TOKEN",
+        "env_var.value": anaconda_token,
+        "env_var.public": "false",
+    }
+    r = requests.post(
+        "{}/repo/{repo_id}/env_vars".format(travis_endpoint, repo_id=repo_id),
+        headers=headers,
+        json=data,
+    )
+    if r.status_code != 201:
+        r.raise_for_status()
+
+
 def travis_cleanup(org, project):
     if os.getenv("GH_TRAVIS_TOKEN"):
         gh = github.Github(os.getenv("GH_TRAVIS_TOKEN"))
diff --git a/conda_smithy/cli.py b/conda_smithy/cli.py
--- a/conda_smithy/cli.py
+++ b/conda_smithy/cli.py
@@ -222,11 +222,9 @@ def __call__(self, args):
             # Assume that the user has enabled travis-ci.com service
             # user-wide or org-wide for all repos
             # ci_register.add_project_to_travis(owner, repo)
-            ci_register.travis_token_update_conda_forge_config(
-                args.feedstock_directory, owner, repo
-            )
             time.sleep(1)
             ci_register.travis_configure(owner, repo)
+            ci_register.add_token_to_travis(owner, repo)
             # Assume that the user has enabled travis-ci.com service
             # user-wide or org-wide for all repos
             # ci_register.travis_cleanup(owner, repo)


Test patch:
Repository: conda-forge/conda-smithy

Issue:
No need to force Azure anymore
The default value of `azure.force` is still True from when Azure Pipelines was being tested.  Since we're no longer testing Azure Pipelines, this can be set back to default False.

From teh gitter:
![image](https://user-images.githubusercontent.com/83846/76534300-4fa6ab80-6447-11ea-8a33-c554d6e1fc39.png)

Sorry for the delay in actually opening this @isuruf .  I have a PR I'm going to push in just a sec.



Discussion:

Resolving patch:
diff --git a/conda_smithy/configure_feedstock.py b/conda_smithy/configure_feedstock.py
--- a/conda_smithy/configure_feedstock.py
+++ b/conda_smithy/configure_feedstock.py
@@ -1330,7 +1330,7 @@ def _load_forge_config(forge_dir, exclusive_config_file):
             # disallow publication of azure artifacts for now.
             "upload_packages": False,
             # Force building all supported providers.
-            "force": True,
+            "force": False,
             # name and id of azure project that the build pipeline is in
             "project_name": "feedstock-builds",
             "project_id": "84710dde-1620-425b-80d0-4cf5baca359d",


Test patch:
Repository: conda-forge/conda-smithy

Issue:
Optionally use `mambabuild`
`conda` error messages are all but uninterpretable. The default advice is to manually try the install command with `mamba` which usually gives much better error messages.

It would be good to have an option to use `conda mambabuild` to short-circuit this process so users can get actionable error messages by default.

From gitter: https://gitter.im/conda-forge/conda-forge.github.io?at=6004a99299549911fc2ace33


Discussion:
@beckermr raised some potential issues which might need to be resolved first:
* https://github.com/mamba-org/boa/issues/110
* https://github.com/mamba-org/boa/issues/100
* https://github.com/mamba-org/boa/issues/102
I fixed 110 and 102, but 100 is more difficult. 

I am willing to look into this _only if_ we want to actually produce builds using `conda mambabuild`.
If we only want `mambabuild` as a debug option then I'd like to skip that one for now.
We can force smithy to disable uploads with mambabuild for now. 

Resolving patch:
diff --git a/conda_smithy/configure_feedstock.py b/conda_smithy/configure_feedstock.py
--- a/conda_smithy/configure_feedstock.py
+++ b/conda_smithy/configure_feedstock.py
@@ -1528,6 +1528,7 @@ def _load_forge_config(forge_dir, exclusive_config_file):
         "conda_forge_output_validation": False,
         "private_upload": False,
         "secrets": [],
+        "build_with_mambabuild": False,
         # Specific channel for package can be given with `${url or channel_alias}::package_name`, defaults to conda-forge channel_alias
         "remote_ci_setup": "conda-forge-ci-setup=3",
     }


Test patch:
Repository: conda-forge/conda-smithy

Issue:
Use ubuntu-latest in azure
Warning from Azure

> Ubuntu 16.04 LTS environment is deprecated and will be removed on September 20, 2021. Migrate to ubuntu-latest instead. For more details, see https://github.com/actions/virtual-environments/issues/3287.


Discussion:

Resolving patch:
diff --git a/conda_smithy/configure_feedstock.py b/conda_smithy/configure_feedstock.py
--- a/conda_smithy/configure_feedstock.py
+++ b/conda_smithy/configure_feedstock.py
@@ -1549,7 +1549,7 @@ def _load_forge_config(forge_dir, exclusive_config_file):
             # default choices for MS-hosted agents
             "settings_linux": {
                 "pool": {
-                    "vmImage": "ubuntu-16.04",
+                    "vmImage": "ubuntu-latest",
                 },
                 "timeoutInMinutes": 360,
             },


Test patch:
Repository: conda-forge/conda-smithy

Issue:
conda-smithy doesn't recognise test/script for outputs
`conda-build` supports test scripts for outputs in the `test/script` key:

https://github.com/conda/conda-build/blob/e458431a7f4becc6d786af6f1a11ee06f64c1513/conda_build/create_test.py#L37-L52

but conda-smithy doesn't recognise this:

https://github.com/conda-forge/conda-smithy/blob/9ab8143a24bc07ea439ef28d6aef71d42955be52/conda_smithy/lint_recipe.py#L51

instead reporting the usual 'This recipe must have some tests'. Can this be added?


Discussion:
See https://github.com/conda-forge/gds-frameio-feedstock/pull/1 for a real-world example of the linter not recognising an output test script.

Resolving patch:
diff --git a/conda_smithy/lint_recipe.py b/conda_smithy/lint_recipe.py
--- a/conda_smithy/lint_recipe.py
+++ b/conda_smithy/lint_recipe.py
@@ -187,6 +187,8 @@ def lintify(meta, recipe_dir=None, conda_forge=False):
                     test_out = get_section(out, "test", lints)
                     if any(key in TEST_KEYS for key in test_out):
                         has_outputs_test = True
+                    elif test_out.get("script", "").endswith((".bat", ".sh")):
+                        has_outputs_test = True
                     else:
                         no_test_hints.append(
                             "It looks like the '{}' output doesn't "


Test patch:
diff --git a/news/output_test_scripts.rst b/news/output_test_scripts.rst
new file mode 100644
--- /dev/null
+++ b/news/output_test_scripts.rst
@@ -0,0 +1,23 @@
+**Added:**
+
+* <news item>
+
+**Changed:**
+
+* <news item>
+
+**Deprecated:**
+
+* <news item>
+
+**Removed:**
+
+* <news item>
+
+**Fixed:**
+
+* correctly detect use of `test/script` in outputs
+
+**Security:**
+
+* <news item>
diff --git a/tests/test_lint_recipe.py b/tests/test_lint_recipe.py
--- a/tests/test_lint_recipe.py
+++ b/tests/test_lint_recipe.py
@@ -171,6 +171,19 @@ def test_test_section(self):
             "It looks like the 'foobar' output doesn't have any tests.", hints
         )
 
+        lints, hints = linter.lintify(
+            {
+                "outputs": [
+                    {"name": "foo", "test": {"script": "test-foo.sh"}},
+                    {"name": "foobar", "test": {"script": "test-foobar.pl"}},
+                ]
+            }
+        )
+        self.assertNotIn(expected_message, lints)
+        self.assertIn(
+            "It looks like the 'foobar' output doesn't have any tests.", hints
+        )
+
     def test_test_section_with_recipe(self):
         # If we have a run_test.py file, we shouldn't need to provide
         # other tests.
Repository: conda-forge/conda-smithy

Issue:
Refine test_on_native_only option in case of multiple cross built platforms
At https://github.com/conda-forge/rust-feedstock/pull/75 we had the (potential) case of cross building for multiple platforms for which only some could be tested via emulation.
The feedstock uses `test_on_native_only: true` because it builds for `osx-arm64` on `osx-64`. If we cross-compiled some Linux platforms, we could make use of QEMU-based tests, but this is prevented due to `test_on_native_only: true`.

We might want a more fine grained configuration, either per target platform or per platform type.
Suggestions thus far (@mbargull/@isuruf in https://github.com/conda-forge/rust-feedstock/pull/75#issuecomment-783604085 ff):
> > Would it make sense to make test_on_native_only optionally a dict platform -> bool to distinguish?
> 
> Yes. I think we can also have an option like,
> 
> ```
> test: <native-only/all/native-and-emulated>
> ```
> 
> and deprecate `test_on_native_only` option. What do you think?



Discussion:
Pointed to here by @chrisburr on Gitter. I am evaluating the very same strategy for Open MPI, so that we can move away from Travis and use Azure to build the `ppc64le` package, yet still being able to run tests via QEMU (no idea how it works ftm). 

Resolving patch:
diff --git a/conda_smithy/configure_feedstock.py b/conda_smithy/configure_feedstock.py
--- a/conda_smithy/configure_feedstock.py
+++ b/conda_smithy/configure_feedstock.py
@@ -1653,6 +1653,8 @@ def _load_forge_config(forge_dir, exclusive_config_file, forge_yml=None):
             "linux_armv7l": None,
             "linux_s390x": None,
         },
+        "test": None,
+        # Following is deprecated
         "test_on_native_only": False,
         "choco": [],
         # Configurable idle timeout.  Used for packages that don't have chatty enough builds
@@ -1761,6 +1763,12 @@ def _load_forge_config(forge_dir, exclusive_config_file, forge_yml=None):
 
     config["secrets"] = sorted(set(config["secrets"] + ["BINSTAR_TOKEN"]))
 
+    if config["test_on_native_only"]:
+        config["test"] = "native_and_emulated"
+
+    if config["test"] is None:
+        config["test"] = "all"
+
     # An older conda-smithy used to have some files which should no longer exist,
     # remove those now.
     old_files = [


Test patch:
Repository: conda-forge/conda-smithy

Issue:
lint for matplotlib vs matplotlib-base
The regex we need is `matplotlib(?!(-base))`


Discussion:

Resolving patch:
diff --git a/conda_smithy/lint_recipe.py b/conda_smithy/lint_recipe.py
--- a/conda_smithy/lint_recipe.py
+++ b/conda_smithy/lint_recipe.py
@@ -734,6 +734,9 @@ def run_conda_forge_specific(meta, recipe_dir, lints, hints):
     package_section = get_section(meta, "package", lints)
     extra_section = get_section(meta, "extra", lints)
     sources_section = get_section(meta, "source", lints)
+    requirements_section = get_section(meta, "requirements", lints)
+    outputs_section = get_section(meta, "outputs", lints)
+
     recipe_dirname = os.path.basename(recipe_dir) if recipe_dir else "recipe"
     recipe_name = package_section.get("name", "").strip()
     is_staged_recipes = recipe_dirname != "recipe"
@@ -838,6 +841,21 @@ def run_conda_forge_specific(meta, recipe_dir, lints, hints):
             if msg not in lints:
                 lints.append(msg)
 
+    # 5: Do not depend on matplotlib, only matplotlib-base
+    run_reqs = requirements_section.get("run") or []
+    for out in outputs_section:
+        run_reqs += (out.get("requirements") or {}).get("run") or []
+    for rq in run_reqs:
+        nm = rq.split(" ")[0].strip()
+        if nm == "matplotlib":
+            msg = (
+                "Recipes should usually depend on `matplotlib-base` as opposed to "
+                "`matplotlib` so that runtime environments do not require large "
+                "packages like `qt`."
+            )
+            if msg not in hints:
+                hints.append(msg)
+
 
 def is_selector_line(line):
     # Using the same pattern defined in conda-build (metadata.py),


Test patch:
diff --git a/tests/test_lint_recipe.py b/tests/test_lint_recipe.py
--- a/tests/test_lint_recipe.py
+++ b/tests/test_lint_recipe.py
@@ -1269,6 +1269,30 @@ def test_build_sh_with_shellcheck_findings(self):
         )
         assert len(hints) < 100
 
+    def test_mpl_base_hint(self):
+        meta = {
+            "requirements": {
+                "run": ["matplotlib >=2.3"],
+            },
+        }
+        lints, hints = linter.lintify(meta, conda_forge=True)
+        expected = "Recipes should usually depend on `matplotlib-base`"
+        self.assertTrue(any(hint.startswith(expected) for hint in hints))
+
+    def test_mpl_base_hint_outputs(self):
+        meta = {
+            "outputs": [
+                {
+                    "requirements": {
+                        "run": ["matplotlib >=2.3"],
+                    },
+                },
+            ],
+        }
+        lints, hints = linter.lintify(meta, conda_forge=True)
+        expected = "Recipes should usually depend on `matplotlib-base`"
+        self.assertTrue(any(hint.startswith(expected) for hint in hints))
+
 
 @pytest.mark.cli
 class TestCLI_recipe_lint(unittest.TestCase):
Repository: conda-forge/conda-smithy

Issue:
`BUILD_WITH_CONDA_DEBUG` not implemented on macOS
<!--
Thanks for reporting your issue.
Please fill out the sections below.
-->

- [X] I read [the conda-forge documentation](https://conda-forge.org/docs/user/introduction.html#how-can-i-install-packages-from-conda-forge) and could not find the solution for my problem there.

Issue:`BUILD_WITH_CONDA_DEBUG`, and so the `--debug` flag of `build_locally.py`, is not implemented on macOS in `run_osx_build.sh`. 

Would a PR that adds that, similar to what is done for Linux in `build_steps.sh`, be welcome ? :)



Discussion:

Resolving patch:
diff --git a/conda_smithy/github.py b/conda_smithy/github.py
--- a/conda_smithy/github.py
+++ b/conda_smithy/github.py
@@ -138,7 +138,7 @@ def create_github_repo(args):
     except GithubException as gh_except:
         if (
             gh_except.data.get("errors", [{}])[0].get("message", "")
-            != u"name already exists on this account"
+            != "name already exists on this account"
         ):
             raise
         gh_repo = user_or_org.get_repo(repo_name)


Test patch: