Repository: conda-forge/conda-smithy

Issue:
BUG: conda-smithy v3.31 mismatches cuda_compiler & image, breaks zip_keys
I'm getting the following rerender in https://github.com/conda-forge/arrow-cpp-feedstock/pull/1325, which mismatches the cuda_compiler with the image, and then predictably fails.
```diff
--- a/.azure-pipelines/azure-pipelines-linux.yml
+++ b/.azure-pipelines/azure-pipelines-linux.yml
@@ -11,7 +11,7 @@ jobs:
       linux_64_cuda_compiler_version11.2:                          # CUDA
         CONFIG: linux_64_cuda_compiler_version11.2
         UPLOAD_PACKAGES: 'True'
-        DOCKER_IMAGE: quay.io/condaforge/linux-anvil-cuda:11.2     # CUDA
+        DOCKER_IMAGE: quay.io/condaforge/linux-anvil-cos7-x86_64   # !! no CUDA !!
       linux_64_cuda_compiler_versionNone:
         CONFIG: linux_64_cuda_compiler_versionNone
         UPLOAD_PACKAGES: 'True'
--- a/.ci_support/linux_64_cuda_compiler_version11.2.yaml
+++ b/.ci_support/linux_64_cuda_compiler_version11.2.yaml           # CUDA
@@ -7,15 +7,15 @@ bzip2:
 cuda_compiler:
-- nvcc                                                             # CUDA
+- None                                                             # !! no CUDA !!
 cuda_compiler_version:
 - '11.2'                                                           # CUDA
 cuda_compiler_version_min:
@@ -23,23 +23,23 @@ cuda_compiler_version_min:
 docker_image:
-- quay.io/condaforge/linux-anvil-cuda:11.2
+- quay.io/condaforge/linux-anvil-cos7-x86_64
 gflags:
--- a/.ci_support/linux_64_cuda_compiler_versionNone.yaml
+++ b/.ci_support/linux_64_cuda_compiler_versionNone.yaml           # no CUDA
@@ -9,13 +9,13 @@ c_compiler:
 cuda_compiler:
-- None                                                             # no CUDA
+- nvcc                                                             # !! CUDA !!
 cuda_compiler_version:
 - None                                                             # no CUDA
 cuda_compiler_version_min:
```

In fact, for 4/4 builds on the arrow feedstock, the rerender randomly mismatches at least two jobs, seemingly by messing up the zip_keys. Not sure what's happening here or whether #1815 had a bug or uncovered another problem.

@mbargull @beckermr 
CC @conda-forge/core


Discussion:
I'm away from my key board. Should be mark it broke? If so can you submit a PR?
It depends a bit what's happening elsewhere and how widespread the breakage is. I'm leaning towards yes, however I'm not 100% that this wouldn't _completely_ break rerendering, because smithy does not rely on the resolver for determining the newest available conda-smithy version, but queries the data directly somehow.

I remember hitting this a while ago (#1491), where rerenders wouldn't work anymore, because smithy would see the new build (even if broken), and fail the version check unless `--no-check-uptodate` is specified. I'm not sure if the conda-forge-admin bot uses that argument, but if it doesn't, marking 3.31 as broken might stop rerenders happening everywhere?
Just tried jaxlib, where a local rerender also messed up python-vs-numpy versions. However, after deleting lines with `cuda_compiler_version` in the meta.yaml and rerendering, the result looked alright (aside from removing all the CUDA builds, obviously).

So at least for CUDA feedstocks, 3.31.0 seems to be pretty badly broken.
> I remember hitting this a while ago (#1491), where rerenders wouldn't work anymore, because smithy would see the new build (even if broken), and fail the version check unless `--no-check-uptodate` is specified. I'm not sure if the conda-forge-admin bot uses that argument, but if it doesn't, marking 3.31 as broken might stop rerenders happening everywhere?

The re-rendering bot [disables the up-to-date check]( https://github.com/conda-forge/webservices-dispatch-action/blob/1d3bcf3994f042633029ce6315292c8420826969/webservices_dispatch_action/rerendering.py#L28 )

There are probably [other places that need to disable update checks]( https://github.com/search?q=org%3Aconda-forge+conda+smithy+rerender+language%3APython&type=code&l=Python )

That said, we may want to rethink how that check works for this reason. Opened issue: https://github.com/conda-forge/conda-smithy/issues/1848
In any event, we should feel comfortable marking it broken if it is not behaving and worry about fixing fallout afterwards
Thank you! I opened https://github.com/conda-forge/admin-requests/pull/947
As another corner-case that's broken in 3.31 (but probably too marginal for a separate issue?) - feedstocks that explicitly set `conda_build_tool: mambabuild` in `conda-forge.yml` (for whatever reason, example: [ray](https://github.com/conda-forge/ray-packages-feedstock/pull/150)), cannot even pass the CI setup because updating the conda installation runs into https://github.com/conda-forge/boa-feedstock/issues/79.
The rendering times for arrow-cpp-feedstock with its multiple outputs is... something.
Rendering https://github.com/conda-forge/arrow-cpp-feedstock/pull/1323/commits/fd8da32e98de34ba3225a2d7e3a064ababe64f9e took more than 10 minutes for me locally.

I've distilled this down to
```yaml
package:
  name: test_conda_smithy_v3.31.0
  version: 1.0

build:
  skip: true  # [not (linux and x86_64) or cuda_compiler_version not in ("None", cuda_compiler_version_min)]

requirements:
  build:
    - {{ compiler("c") }}
    - {{ compiler("cuda") }}  # [cuda_compiler_version != "None"]

outputs:
  - name: test_conda_smithy_v3.31.0-output
    requirements:
      build:
        - {{ compiler("c") }}
        - {{ compiler("cuda") }}  # [cuda_compiler_version != "None"]

about: {}
```
which (indeterministically) fails after https://github.com/conda-forge/conda-smithy/pull/1815/commits/85482fdd1ccf0c1c8a087f926d7c37ee4d1cae63 .

I'll take a look later today -- if I can't figure this out right away, then we can just revert that commit for now.
But we should add the above test case in any case, of course.

----
Thanks for reporting this quickly and handling the mark as `broken` stuff!
Awesome that you reduced this, thanks for tackling this so quickly!

> The rendering times for arrow-cpp-feedstock with its multiple outputs is... something.
> Rendering https://github.com/conda-forge/arrow-cpp-feedstock/commit/fd8da32e98de34ba3225a2d7e3a064ababe64f9e took more than 10 minutes for me locally.

Yeah... Local rerenders take up to 25min for me now - not fun. 😑 
PYTHONHASHSEED=4 will cause the error for those trying this out.

Resolving patch:
diff --git a/conda_smithy/configure_feedstock.py b/conda_smithy/configure_feedstock.py
--- a/conda_smithy/configure_feedstock.py
+++ b/conda_smithy/configure_feedstock.py
@@ -6,6 +6,7 @@
 import re
 import sys
 import subprocess
+import pprint
 import textwrap
 import time
 import yaml
@@ -296,6 +297,119 @@ def _trim_unused_pin_run_as_build(all_used_vars):
         del all_used_vars["pin_run_as_build"]
 
 
+def _get_used_key_values_by_input_order(
+    squished_input_variants,
+    squished_used_variants,
+    all_used_vars,
+):
+    used_key_values = {
+        key: squished_input_variants[key]
+        for key in all_used_vars
+        if key in squished_input_variants
+    }
+    logger.debug(
+        "initial used_key_values {}".format(pprint.pformat(used_key_values))
+    )
+
+    # we want remove any used key values not in used variants and make sure they follow the
+    #   input order
+    # zipped keys are a special case since they are ordered by the list of tuple of zipped
+    #   key values
+    # so we do the zipped keys first and then do the rest
+    zipped_tuples = {}
+    zipped_keys = set()
+    for keyset in squished_input_variants["zip_keys"]:
+        zipped_tuples[tuple(keyset)] = list(
+            zip(*[squished_input_variants[k] for k in keyset])
+        )
+        zipped_keys |= set(keyset)
+    logger.debug("zipped_keys {}".format(pprint.pformat(zipped_keys)))
+    logger.debug("zipped_tuples {}".format(pprint.pformat(zipped_tuples)))
+
+    for keyset, tuples in zipped_tuples.items():
+        # for each set of zipped keys from squished_input_variants,
+        # we trim them down to what is in squished_used_variants
+        used_keyset = []
+        used_keyset_inds = []
+        for k in keyset:
+            if k in squished_used_variants:
+                used_keyset.append(k)
+                used_keyset_inds.append(keyset.index(k))
+        used_keyset = tuple(used_keyset)
+        used_keyset_inds = tuple(used_keyset_inds)
+
+        # if we find nothing, keep going
+        if not used_keyset:
+            continue
+
+        # this trims the zipped tuples down to the used keys
+        used_tuples = tuple(
+            [
+                tuple(
+                    [
+                        tup[used_keyset_ind]
+                        for used_keyset_ind in used_keyset_inds
+                    ]
+                )
+                for tup in tuples
+            ]
+        )
+        logger.debug("used_keyset {}".format(pprint.pformat(used_keyset)))
+        logger.debug(
+            "used_keyset_inds {}".format(pprint.pformat(used_keyset_inds))
+        )
+        logger.debug("used_tuples {}".format(pprint.pformat(used_tuples)))
+
+        # this is the set of tuples that we want to keep, but need to be reordered
+        used_tuples_to_be_reordered = set(
+            list(zip(*[squished_used_variants[k] for k in used_keyset]))
+        )
+        logger.debug(
+            "used_tuples_to_be_reordered {}".format(
+                pprint.pformat(used_tuples_to_be_reordered)
+            )
+        )
+
+        # we double check the logic above by looking to ensure everything in
+        #   the squished_used_variants
+        # is in the squished_input_variants
+        used_tuples_set = set(used_tuples)
+        logger.debug(
+            "are all used tuples in input tuples? %s",
+            all(
+                used_tuple in used_tuples_set
+                for used_tuple in used_tuples_to_be_reordered
+            ),
+        )
+
+        # now we do the final rdering
+        final_used_tuples = tuple(
+            [tup for tup in used_tuples if tup in used_tuples_to_be_reordered]
+        )
+        logger.debug(
+            "final_used_tuples {}".format(pprint.pformat(final_used_tuples))
+        )
+
+        # now we reconstruct the list of values per key and replace in used_key_values
+        # we keep only keys in all_used_vars
+        for i, k in enumerate(used_keyset):
+            if k in all_used_vars:
+                used_key_values[k] = [tup[i] for tup in final_used_tuples]
+
+    # finally, we handle the rest of the keys that are not zipped
+    for k, v in squished_used_variants.items():
+        if k in all_used_vars and k not in zipped_keys:
+            used_key_values[k] = v
+
+    logger.debug(
+        "post input reorder used_key_values {}".format(
+            pprint.pformat(used_key_values)
+        )
+    )
+
+    return used_key_values
+
+
 def _collapse_subpackage_variants(
     list_of_metas, root_path, platform, arch, forge_config
 ):
@@ -322,7 +436,9 @@ def _collapse_subpackage_variants(
         # smithy CI support scripts
         # future MPI variants have to be added here
         if "mpi" in all_used_vars:
-            all_used_vars.update(["mpich", "openmpi"])
+            all_used_vars.update(
+                ["mpich", "openmpi", "msmpi", "mpi_serial", "impi"]
+            )
         all_variants.update(
             conda_build.utils.HashableDict(v) for v in meta.config.variants
         )
@@ -339,6 +455,10 @@ def _collapse_subpackage_variants(
     if "target_platform" in all_used_vars:
         top_level_loop_vars.add("target_platform")
 
+    logger.debug(
+        "initial all_used_vars {}".format(pprint.pformat(all_used_vars))
+    )
+
     # this is the initial collection of all variants before we discard any.  "Squishing"
     #     them is necessary because the input form is already broken out into one matrix
     #     configuration per item, and we want a single dict, with each key representing many values
@@ -350,10 +470,23 @@ def _collapse_subpackage_variants(
     squished_used_variants = (
         conda_build.variants.list_of_dicts_to_dict_of_lists(list(all_variants))
     )
+    logger.debug(
+        "squished_input_variants {}".format(
+            pprint.pformat(squished_input_variants)
+        )
+    )
+    logger.debug(
+        "squished_used_variants {}".format(
+            pprint.pformat(squished_used_variants)
+        )
+    )
 
     # these are variables that only occur in the top level, and thus won't show up as loops in the
     #     above collection of all variants.  We need to transfer them from the input_variants.
     preserve_top_level_loops = set(top_level_loop_vars) - set(all_used_vars)
+    logger.debug(
+        "preserve_top_level_loops {}".format(preserve_top_level_loops)
+    )
 
     # Add in some variables that should always be preserved
     always_keep_keys = {
@@ -383,25 +516,19 @@ def _collapse_subpackage_variants(
     all_used_vars.update(always_keep_keys)
     all_used_vars.update(top_level_vars)
 
-    used_key_values = {
-        key: squished_input_variants[key]
-        for key in all_used_vars
-        if key in squished_input_variants
-    }
+    logger.debug(
+        "final all_used_vars {}".format(pprint.pformat(all_used_vars))
+    )
+    logger.debug("top_level_vars {}".format(pprint.pformat(top_level_vars)))
+    logger.debug(
+        "top_level_loop_vars {}".format(pprint.pformat(top_level_loop_vars))
+    )
 
-    for k, v in squished_used_variants.items():
-        if k in all_used_vars:
-            input_variant = squished_input_variants.get(k)
-            if input_variant and isinstance(input_variant, (tuple, list)):
-                # Ensure we retain the input order to avoid mismatched entries
-                # with the preserve_top_level_loops overrides below.
-                # NOTE: "pin_run_as_build" (dict of dict) order is not handled.
-                # "tuple" below only used for uniform list/tuple/str comparison.
-                v_set = set(map(tuple, v))
-                v = type(input_variant)(
-                    v_i for v_i in input_variant if tuple(v_i) in v_set
-                )
-            used_key_values[k] = v
+    used_key_values = _get_used_key_values_by_input_order(
+        squished_input_variants,
+        squished_used_variants,
+        all_used_vars,
+    )
 
     for k in preserve_top_level_loops:
         used_key_values[k] = squished_input_variants[k]
@@ -425,8 +552,9 @@ def _collapse_subpackage_variants(
     _trim_unused_zip_keys(used_key_values)
     _trim_unused_pin_run_as_build(used_key_values)
 
-    logger.debug("top_level_loop_vars {}".format(top_level_loop_vars))
-    logger.debug("used_key_values {}".format(used_key_values))
+    logger.debug(
+        "final used_key_values {}".format(pprint.pformat(used_key_values))
+    )
 
     return (
         break_up_top_level_values(top_level_loop_vars, used_key_values),
@@ -493,6 +621,9 @@ def dump_subspace_config_files(
         arch,
         forge_config,
     )
+    logger.debug(
+        "collapsed subspace config files: {}".format(pprint.pformat(configs))
+    )
 
     # get rid of the special object notation in the yaml file for objects that we dump
     yaml.add_representer(set, yaml.representer.SafeRepresenter.represent_list)
@@ -523,6 +654,9 @@ def dump_subspace_config_files(
             os.makedirs(out_folder)
 
         config = finalize_config(config, platform, arch, forge_config)
+        logger.debug(
+            "finalized config file: {}".format(pprint.pformat(config))
+        )
 
         with write_file(out_path) as f:
             yaml.dump(config, f, default_flow_style=False)


Test patch:
diff --git a/tests/recipes/variant_mismatches/conda_build_config.yaml b/tests/recipes/variant_mismatches/conda_build_config.yaml
new file mode 100644
--- /dev/null
+++ b/tests/recipes/variant_mismatches/conda_build_config.yaml
@@ -0,0 +1,13 @@
+a:
+  - 1
+  - 2
+  - 2
+b:
+  - 1
+  - 2
+  - 1
+
+zip_keys:
+  -
+    - a
+    - b
diff --git a/tests/recipes/variant_mismatches/meta.yaml b/tests/recipes/variant_mismatches/meta.yaml
new file mode 100644
--- /dev/null
+++ b/tests/recipes/variant_mismatches/meta.yaml
@@ -0,0 +1,8 @@
+package:
+  name: test-variant-mismatches
+  version: 1.0
+
+build:
+  skip: true  # [not linux64 or a != b]
+
+about: {}
diff --git a/tests/test_cli.py b/tests/test_cli.py
--- a/tests/test_cli.py
+++ b/tests/test_cli.py
@@ -328,3 +328,45 @@ def test_regenerate(py_recipe, testing_workdir):
 
     # one py ver, no target_platform  (tests that older configs don't stick around)
     assert len(os.listdir(matrix_folder)) == 4
+
+
+def test_render_variant_mismatches(testing_workdir):
+    parser = argparse.ArgumentParser()
+    subparser = parser.add_subparsers()
+    init_obj = cli.Init(subparser)
+    regen_obj = cli.Regenerate(subparser)
+    _thisdir = os.path.abspath(os.path.dirname(__file__))
+    recipe = os.path.join(_thisdir, "recipes", "variant_mismatches")
+    feedstock_dir = os.path.join(
+        testing_workdir, "test-variant-mismatches-feedstock"
+    )
+    args = InitArgs(
+        recipe_directory=recipe,
+        feedstock_directory=feedstock_dir,
+        temporary_directory=os.path.join(recipe, "temp"),
+    )
+    init_obj(args)
+    # Ignore conda-forge-pinning for this test, as the test relies on conda-forge-pinning
+    # not being present
+    args = RegenerateArgs(
+        feedstock_directory=feedstock_dir,
+        feedstock_config=None,
+        commit=False,
+        no_check_uptodate=True,
+        exclusive_config_file="recipe/conda_build_config.yaml",
+        check=False,
+        temporary_directory=os.path.join(recipe, "temp"),
+    )
+    regen_obj(args)
+
+    matrix_dir = os.path.join(feedstock_dir, ".ci_support")
+    cfgs = os.listdir(matrix_dir)
+    assert len(cfgs) == 3  # readme + 2 configs
+
+    for _cfg in cfgs:
+        if _cfg == "README":
+            continue
+        cfg = os.path.join(matrix_dir, _cfg)
+        with open(cfg, "r") as f:
+            data = yaml.safe_load(f)
+        assert data["a"] == data["b"]
diff --git a/tests/test_configure_feedstock.py b/tests/test_configure_feedstock.py
--- a/tests/test_configure_feedstock.py
+++ b/tests/test_configure_feedstock.py
@@ -943,3 +943,935 @@ def test_remote_ci_setup(config_yaml):
             '"py-lief<0.12"',
         ]
     )
+
+
+@pytest.mark.parametrize(
+    "squished_input_variants,squished_used_variants,all_used_vars,used_key_values",
+    [
+        (
+            dict(
+                [
+                    (
+                        "extend_keys",
+                        {
+                            "extend_keys",
+                            "ignore_build_only_deps",
+                            "ignore_version",
+                            "pin_run_as_build",
+                        },
+                    ),
+                    ("ignore_build_only_deps", {"python", "numpy"}),
+                    (
+                        "pin_run_as_build",
+                        dict(
+                            [
+                                (
+                                    "python",
+                                    {"max_pin": "x.x", "min_pin": "x.x"},
+                                ),
+                                (
+                                    "r-base",
+                                    {"max_pin": "x.x", "min_pin": "x.x"},
+                                ),
+                                ("flann", {"max_pin": "x.x.x"}),
+                                ("graphviz", {"max_pin": "x"}),
+                                ("libsvm", {"max_pin": "x"}),
+                                ("netcdf-cxx4", {"max_pin": "x.x"}),
+                                ("occt", {"max_pin": "x.x"}),
+                                ("poppler", {"max_pin": "x.x"}),
+                                ("vlfeat", {"max_pin": "x.x.x"}),
+                            ]
+                        ),
+                    ),
+                    ("glew", ["2.1"]),
+                    ("pango", ["1.50"]),
+                    ("libgoogle_cloud_compute_devel", ["2.21"]),
+                    ("dav1d", ["1.2.1"]),
+                    ("lzo", ["2"]),
+                    ("pybind11_abi", ["4"]),
+                    ("sqlite", ["3"]),
+                    ("nlopt", ["2.7"]),
+                    ("aws_checksums", ["0.1.18"]),
+                    ("aws_c_mqtt", ["0.10.2"]),
+                    ("aws_c_s3", ["0.5.1"]),
+                    ("libaec", ["1"]),
+                    ("pari", ["2.15.* *_pthread"]),
+                    ("libgoogle_cloud_devel", ["2.21"]),
+                    ("libjpeg_turbo", ["3"]),
+                    ("libblitz", ["1.0.2"]),
+                    ("ptscotch", ["7.0.4"]),
+                    ("openslide", ["4"]),
+                    ("libdeflate", ["1.19"]),
+                    ("gf2x", ["1.3"]),
+                    ("cuda_compiler_version_min", ["11.2"]),
+                    ("aws_c_event_stream", ["0.4.2"]),
+                    ("arb", ["2.23"]),
+                    ("expat", ["2"]),
+                    ("bzip2", ["1"]),
+                    ("target_goexe", [""]),
+                    ("VERBOSE_AT", ["V=1"]),
+                    ("cran_mirror", ["https://cran.r-project.org"]),
+                    ("aws_c_compression", ["0.2.18"]),
+                    ("libsvm", ["332"]),
+                    ("uhd", ["4.6.0"]),
+                    ("libcrc32c", ["1.1"]),
+                    ("libtensorflow_cc", ["2.15"]),
+                    ("msgpack_cxx", ["6"]),
+                    ("libunwind", ["1.6"]),
+                    ("libv8", ["8.9.83"]),
+                    ("libpq", ["16"]),
+                    ("lmdb", ["0.9.29"]),
+                    ("root_base", ["6.28.10", "6.30.2"]),
+                    ("geotiff", ["1.7.1"]),
+                    ("google_cloud_cpp", ["2.21"]),
+                    ("mpfr", ["4"]),
+                    ("gmp", ["6"]),
+                    ("harfbuzz", ["8"]),
+                    ("_libgcc_mutex", ["0.1 conda_forge"]),
+                    ("wxwidgets", ["3.2"]),
+                    ("svt_av1", ["1.8.0"]),
+                    ("libcurl", ["8"]),
+                    ("liblapack", ["3.9 *netlib"]),
+                    ("libmatio_cpp", ["0.2.3"]),
+                    ("libvips", ["8"]),
+                    ("aws_sdk_cpp", ["1.11.267"]),
+                    ("proj", ["9.3.1"]),
+                    ("libgoogle_cloud_all_devel", ["2.21"]),
+                    ("libgoogle_cloud_aiplatform_devel", ["2.21"]),
+                    ("libpcap", ["1.10"]),
+                    ("pulseaudio_client", ["16.1"]),
+                    ("ffmpeg", ["6"]),
+                    ("graphviz", ["9"]),
+                    ("cudnn", ["8"]),
+                    ("x264", ["1!164.*"]),
+                    ("libcint", ["5.5"]),
+                    ("s2n", ["1.4.4"]),
+                    ("sdl2_ttf", ["2"]),
+                    ("pugixml", ["1.14"]),
+                    ("flatbuffers", ["23.5.26"]),
+                    ("krb5", ["1.20"]),
+                    ("ntl", ["11.4.3"]),
+                    ("libev", ["4.33"]),
+                    ("aws_c_io", ["0.14.4"]),
+                    ("target_gobin", ["${PREFIX}/bin/"]),
+                    ("cxx_compiler", ["gxx"]),
+                    ("ccr", ["1.3"]),
+                    ("postgresql", ["16"]),
+                    ("antic", ["0.2"]),
+                    ("gst_plugins_base", ["1.22"]),
+                    ("rdma_core", ["49"]),
+                    ("libssh2", ["1"]),
+                    ("libtiff", ["4.6"]),
+                    ("kealib", ["1.5"]),
+                    ("mpich", ["4"]),
+                    ("spdlog", ["1.12"]),
+                    ("netcdf_fortran", ["4.6"]),
+                    ("elfutils", ["0.190"]),
+                    ("gsl", ["2.7"]),
+                    ("eclib", ["20231211"]),
+                    ("tiledb", ["2.20"]),
+                    ("libhugetlbfs", ["2"]),
+                    ("nss", ["3"]),
+                    ("netcdf_cxx4", ["4.3"]),
+                    ("pcl", ["1.13.1"]),
+                    ("sox", ["14.4.2"]),
+                    ("superlu_dist", ["8"]),
+                    ("libgoogle_cloud_dialogflow_es_devel", ["2.21"]),
+                    ("attr", ["2.5"]),
+                    ("log4cxx", ["1.2.0"]),
+                    ("aws_c_sdkutils", ["0.1.15"]),
+                    ("libxml2", ["2"]),
+                    ("libsentencepiece", ["0.1.99"]),
+                    ("tbb_devel", ["2021"]),
+                    ("jsoncpp", ["1.9.5"]),
+                    ("msgpack_c", ["6"]),
+                    ("libzip", ["1"]),
+                    ("mkl", ["2023"]),
+                    ("giflib", ["5.2"]),
+                    ("fortran_compiler", ["gfortran"]),
+                    ("libboost_python_devel", ["1.82"]),
+                    ("dbus", ["1"]),
+                    ("libwebp_base", ["1"]),
+                    ("libflint", ["2.9"]),
+                    ("libsoup", ["3"]),
+                    ("libgoogle_cloud_bigtable_devel", ["2.21"]),
+                    ("abseil_cpp", ["20220623.0"]),
+                    ("librdkafka", ["2.2"]),
+                    ("starlink_ast", ["9.2.7"]),
+                    ("lerc", ["4"]),
+                    ("davix", ["0.8"]),
+                    ("libxsmm", ["1"]),
+                    ("poppler", ["23.07"]),
+                    ("dcap", ["2.47"]),
+                    ("gnuradio_core", ["3.10.9"]),
+                    ("gsoap", ["2.8.123"]),
+                    ("fmt", ["10"]),
+                    ("libopencv", ["4.9.0"]),
+                    ("aws_c_http", ["0.8.1"]),
+                    ("lua", ["5"]),
+                    ("qt", ["5.15"]),
+                    ("pcre", ["8"]),
+                    ("ldas_tools_framecpp", ["2.9"]),
+                    ("ruby", ["2.5", "2.6"]),
+                    ("libgoogle_cloud_policytroubleshooter_devel", ["2.21"]),
+                    ("libdap4", ["3.20.6"]),
+                    ("mimalloc", ["2.1.2"]),
+                    ("coin_or_osi", ["0.108"]),
+                    ("gdk_pixbuf", ["2"]),
+                    ("target_goarch", ["amd64"]),
+                    ("assimp", ["5.3.1"]),
+                    ("flann", ["1.9.2"]),
+                    ("gct", ["6.2.1629922860"]),
+                    ("aws_c_auth", ["0.7.16"]),
+                    ("pyqtchart", ["5.15"]),
+                    ("coin_or_clp", ["1.17"]),
+                    ("grpc_cpp", ["1.52"]),
+                    ("libgdal", ["3.8"]),
+                    ("volk", ["3.1"]),
+                    ("curl", ["8"]),
+                    ("geos", ["3.12.1"]),
+                    ("zstd", ["1.5"]),
+                    ("libabseil", ["20240116"]),
+                    ("blas_impl", ["openblas", "mkl", "blis"]),
+                    ("openblas", ["0.3.*"]),
+                    ("zlib_ng", ["2.0"]),
+                    ("liblapacke", ["3.9 *netlib"]),
+                    ("libframel", ["8.41"]),
+                    ("postgresql_plpython", ["16"]),
+                    ("libgoogle_cloud", ["2.21"]),
+                    ("gstreamer", ["1.22"]),
+                    ("lcms", ["2"]),
+                    ("libuuid", ["2"]),
+                    ("coin_or_cgl", ["0.60"]),
+                    ("rust_compiler", ["rust"]),
+                    ("bullet_cpp", ["3.25"]),
+                    ("fftw", ["3"]),
+                    ("tbb", ["2021"]),
+                    ("libsecret", ["0.18"]),
+                    ("libgoogle_cloud_spanner_devel", ["2.21"]),
+                    ("channel_sources", ["conda-forge"]),
+                    ("pcre2", ["10.42"]),
+                    ("glib", ["2"]),
+                    ("google_cloud_cpp_common", ["0.25.0"]),
+                    ("openmpi", ["4"]),
+                    ("ncurses", ["6"]),
+                    ("openh264", ["2.4.1"]),
+                    ("wcslib", ["8"]),
+                    ("soapysdr", ["0.8"]),
+                    ("tinyxml2", ["10"]),
+                    ("libavif", ["1.0.1"]),
+                    ("cairo", ["1"]),
+                    ("icu", ["73"]),
+                    ("libsndfile", ["1.2"]),
+                    ("aom", ["3.7"]),
+                    ("x265", ["3.5"]),
+                    ("jasper", ["4"]),
+                    ("pyqt", ["5.15"]),
+                    ("xrootd", ["5"]),
+                    ("gnutls", ["3.7"]),
+                    ("rocksdb", ["8.0"]),
+                    ("libgit2", ["1.7"]),
+                    ("libgoogle_cloud_oauth2_devel", ["2.21"]),
+                    ("zfp", ["1.0"]),
+                    ("libtorch", ["2.1"]),
+                    ("libarchive", ["3.7"]),
+                    ("freetype", ["2"]),
+                    ("poco", ["1.13.2"]),
+                    ("libthrift", ["0.19.0"]),
+                    ("qt_main", ["5.15"]),
+                    ("libosqp", ["0.6.3"]),
+                    ("ucx", ["1.15.0"]),
+                    ("libgoogle_cloud_automl_devel", ["2.21"]),
+                    ("slepc", ["3.20"]),
+                    ("aws_c_cal", ["0.6.10"]),
+                    ("p11_kit", ["0.24"]),
+                    ("target_platform", ["linux-64"]),
+                    ("libblas", ["3.9 *netlib"]),
+                    ("libgoogle_cloud_bigquery_devel", ["2.21"]),
+                    ("libmicrohttpd", ["1.0"]),
+                    ("mpg123", ["1.32"]),
+                    ("srm_ifce", ["1.24.6"]),
+                    ("petsc", ["3.20"]),
+                    ("libiconv", ["1"]),
+                    ("aws_crt_cpp", ["0.26.2"]),
+                    ("console_bridge", ["1.0"]),
+                    ("libgoogle_cloud_storage_devel", ["2.21"]),
+                    ("lz4_c", ["1.9.3"]),
+                    ("mumps_mpi", ["5.6.2"]),
+                    ("coincbc", ["2.10"]),
+                    ("orc", ["1.9.2"]),
+                    ("libcblas", ["3.9 *netlib"]),
+                    ("readline", ["8"]),
+                    ("nodejs", ["18", "20"]),
+                    ("glpk", ["5.0"]),
+                    ("imath", ["3.1.9"]),
+                    ("gdal", ["3.8"]),
+                    ("nettle", ["3.9"]),
+                    ("qtkeychain", ["0.14"]),
+                    ("c_ares", ["1"]),
+                    ("libduckdb_devel", ["0.9.2"]),
+                    ("occt", ["7.7.2"]),
+                    ("qt6_main", ["6.6"]),
+                    ("perl", ["5.32.1"]),
+                    ("libidn2", ["2"]),
+                    ("pyqtwebengine", ["5.15"]),
+                    ("coin_or_utils", ["2.11"]),
+                    ("libopenvino_dev", ["2023.3.0"]),
+                    ("googleapis_cpp", ["0.10"]),
+                    ("libwebp", ["1"]),
+                    ("coin_or_cbc", ["2.10"]),
+                    ("channel_targets", ["conda-forge main"]),
+                    ("sdl2_image", ["2"]),
+                    ("sdl2_mixer", ["2"]),
+                    ("vtk", ["9.2.6"]),
+                    ("librsvg", ["2"]),
+                    ("jpeg", ["9"]),
+                    ("hdf4", ["4.2.15"]),
+                    ("pytorch", ["2.1"]),
+                    ("libintervalxt", ["3"]),
+                    ("thrift_cpp", ["0.19.0"]),
+                    ("libgoogle_cloud_discoveryengine_devel", ["2.21"]),
+                    ("arpack", ["3.8"]),
+                    ("libtensorflow", ["2.15"]),
+                    ("vlfeat", ["0.9.21"]),
+                    ("snappy", ["1"]),
+                    ("capnproto", ["0.10.2"]),
+                    ("libmatio", ["1.5.26"]),
+                    ("c_compiler", ["gcc"]),
+                    ("cgo_compiler", ["go-cgo"]),
+                    ("ipopt", ["3.14.14"]),
+                    ("libiio", ["0"]),
+                    ("singular", ["4.3.2.p8"]),
+                    ("libhwy", ["1.0"]),
+                    ("zeromq", ["4.3.5"]),
+                    ("pixman", ["0"]),
+                    ("libspatialindex", ["1.9.3"]),
+                    ("libffi", ["3.4"]),
+                    ("nspr", ["4"]),
+                    ("petsc4py", ["3.20"]),
+                    ("libsqlite", ["3"]),
+                    ("pulseaudio", ["16.1"]),
+                    ("libopentelemetry_cpp", ["1.14"]),
+                    ("libptscotch", ["7.0.4"]),
+                    ("libraw", ["0.21"]),
+                    ("libgoogle_cloud_dlp_devel", ["2.21"]),
+                    ("libgoogle_cloud_iam_devel", ["2.21"]),
+                    ("openjpeg", ["2"]),
+                    ("libhwloc", ["2.9.1"]),
+                    ("r_base", ["4.2", "4.3"]),
+                    ("gfal2", ["2.21"]),
+                    ("libexactreal", ["4"]),
+                    ("mkl_devel", ["2023"]),
+                    ("zlib", ["1.2"]),
+                    ("libmed", ["4.1"]),
+                    ("fontconfig", ["2"]),
+                    ("xz", ["5"]),
+                    ("suitesparse", ["5"]),
+                    ("libgoogle_cloud_speech_devel", ["2.21"]),
+                    ("sdl2_net", ["2"]),
+                    ("slepc4py", ["3.20"]),
+                    ("tk", ["8.6"]),
+                    ("libpng", ["1.6"]),
+                    ("libssh", ["0.10"]),
+                    ("urdfdom", ["3.1"]),
+                    ("metis", ["5.1.0"]),
+                    ("libnetcdf", ["4.9.2"]),
+                    ("sdl2", ["2"]),
+                    ("target_goos", ["linux"]),
+                    ("cfitsio", ["4.3.0"]),
+                    ("pulseaudio_daemon", ["16.1"]),
+                    ("mumps_seq", ["5.6.2"]),
+                    ("hdf5", ["1.14.3"]),
+                    ("nccl", ["2"]),
+                    ("libevent", ["2.1.12"]),
+                    ("exiv2", ["0.27"]),
+                    ("libeantic", ["2"]),
+                    ("alsa_lib", ["1.2.10"]),
+                    ("glog", ["0.7"]),
+                    ("libscotch", ["7.0.4"]),
+                    ("cutensor", ["2"]),
+                    ("json_c", ["0.17"]),
+                    ("aws_c_common", ["0.9.13"]),
+                    ("isl", ["0.26"]),
+                    ("openssl", ["3"]),
+                    ("xerces_c", ["3.2"]),
+                    ("cpu_optimization_target", ["nocona"]),
+                    ("libflatsurf", ["3"]),
+                    ("libkml", ["1.3"]),
+                    ("gflags", ["2.2"]),
+                    ("libboost_devel", ["1.82"]),
+                    ("libgoogle_cloud_dialogflow_cx_devel", ["2.21"]),
+                    ("VERBOSE_CM", ["VERBOSE=1"]),
+                    ("openexr", ["3.2"]),
+                    ("scotch", ["7.0.4"]),
+                    ("libgoogle_cloud_pubsub_devel", ["2.21"]),
+                    ("libabseil_static", ["20220623.0"]),
+                    ("go_compiler", ["go-nocgo"]),
+                    ("re2", ["2023.06.02"]),
+                    ("tensorflow", ["2.15"]),
+                    ("libarrow_all", ("12", "14", "13", "15")),
+                    ("arrow_cpp", ("12", "14", "13", "15")),
+                    ("libarrow", ("12", "14", "13", "15")),
+                    ("c_compiler_version", ("12", "11", "10", "12")),
+                    ("fortran_compiler_version", ("12", "11", "10", "12")),
+                    ("cdt_name", ("cos7", "cos7", "cos7", "cos6")),
+                    (
+                        "docker_image",
+                        (
+                            "quay.io/condaforge/linux-anvil-cos7-x86_64",
+                            "quay.io/condaforge/linux-anvil-cuda:11.8",
+                            "quay.io/condaforge/linux-anvil-cuda:11.2",
+                            "quay.io/condaforge/linux-anvil-cos7-x86_64",
+                        ),
+                    ),
+                    ("cuda_compiler", ("cuda-nvcc", "nvcc", "nvcc", "None")),
+                    ("cxx_compiler_version", ("12", "11", "10", "12")),
+                    (
+                        "cuda_compiler_version",
+                        ("12.0", "11.8", "11.2", "None"),
+                    ),
+                    ("libgrpc", ("1.61",)),
+                    ("libprotobuf", ("4.25.2",)),
+                    ("c_stdlib_version", ("2.12",)),
+                    ("c_stdlib", ("sysroot",)),
+                    (
+                        "python",
+                        (
+                            "3.9.* *_cpython",
+                            "3.10.* *_cpython",
+                            "3.11.* *_cpython",
+                            "3.8.* *_cpython",
+                            "3.12.* *_cpython",
+                        ),
+                    ),
+                    ("numpy", ("1.22", "1.22", "1.23", "1.22", "1.26")),
+                    (
+                        "python_impl",
+                        (
+                            "cpython",
+                            "cpython",
+                            "cpython",
+                            "cpython",
+                            "cpython",
+                        ),
+                    ),
+                    (
+                        "zip_keys",
+                        [
+                            ["python", "numpy", "python_impl"],
+                            ["arrow_cpp", "libarrow", "libarrow_all"],
+                            ["c_stdlib", "c_stdlib_version"],
+                            [
+                                "c_compiler_version",
+                                "cxx_compiler_version",
+                                "fortran_compiler_version",
+                                "cuda_compiler",
+                                "cuda_compiler_version",
+                                "cdt_name",
+                                "docker_image",
+                            ],
+                            ["libgrpc", "libprotobuf"],
+                        ],
+                    ),
+                ]
+            ),
+            dict(
+                [
+                    (
+                        "extend_keys",
+                        {
+                            "extend_keys",
+                            "ignore_build_only_deps",
+                            "ignore_version",
+                            "pin_run_as_build",
+                        },
+                    ),
+                    ("ignore_build_only_deps", {"python", "numpy"}),
+                    (
+                        "pin_run_as_build",
+                        dict(
+                            [
+                                (
+                                    "python",
+                                    {"max_pin": "x.x", "min_pin": "x.x"},
+                                ),
+                                (
+                                    "r-base",
+                                    {"max_pin": "x.x", "min_pin": "x.x"},
+                                ),
+                                ("flann", {"max_pin": "x.x.x"}),
+                                ("graphviz", {"max_pin": "x"}),
+                                ("libsvm", {"max_pin": "x"}),
+                                ("netcdf-cxx4", {"max_pin": "x.x"}),
+                                ("occt", {"max_pin": "x.x"}),
+                                ("poppler", {"max_pin": "x.x"}),
+                                ("vlfeat", {"max_pin": "x.x.x"}),
+                            ]
+                        ),
+                    ),
+                    ("glew", ["2.1"]),
+                    ("pango", ["1.50"]),
+                    ("libgoogle_cloud_compute_devel", ["2.21"]),
+                    ("dav1d", ["1.2.1"]),
+                    ("lzo", ["2"]),
+                    ("pybind11_abi", ["4"]),
+                    ("sqlite", ["3"]),
+                    ("nlopt", ["2.7"]),
+                    ("aws_checksums", ["0.1.18"]),
+                    ("aws_c_mqtt", ["0.10.2"]),
+                    ("aws_c_s3", ["0.5.1"]),
+                    ("libaec", ["1"]),
+                    ("pari", ["2.15.* *_pthread"]),
+                    ("libgoogle_cloud_devel", ["2.21"]),
+                    ("libjpeg_turbo", ["3"]),
+                    ("libblitz", ["1.0.2"]),
+                    ("ptscotch", ["7.0.4"]),
+                    ("openslide", ["4"]),
+                    ("libdeflate", ["1.19"]),
+                    ("gf2x", ["1.3"]),
+                    ("cuda_compiler_version_min", ["11.2"]),
+                    ("aws_c_event_stream", ["0.4.2"]),
+                    ("arb", ["2.23"]),
+                    ("expat", ["2"]),
+                    ("bzip2", ["1"]),
+                    ("target_goexe", [""]),
+                    ("VERBOSE_AT", ["V=1"]),
+                    ("cran_mirror", ["https://cran.r-project.org"]),
+                    ("aws_c_compression", ["0.2.18"]),
+                    ("libsvm", ["332"]),
+                    ("uhd", ["4.6.0"]),
+                    ("libcrc32c", ["1.1"]),
+                    ("libtensorflow_cc", ["2.15"]),
+                    ("msgpack_cxx", ["6"]),
+                    ("libunwind", ["1.6"]),
+                    ("libv8", ["8.9.83"]),
+                    ("libpq", ["16"]),
+                    ("lmdb", ["0.9.29"]),
+                    ("root_base", ["6.28.10", "6.30.2"]),
+                    ("geotiff", ["1.7.1"]),
+                    ("google_cloud_cpp", ["2.21"]),
+                    ("mpfr", ["4"]),
+                    ("gmp", ["6"]),
+                    ("harfbuzz", ["8"]),
+                    ("_libgcc_mutex", ["0.1 conda_forge"]),
+                    ("wxwidgets", ["3.2"]),
+                    ("svt_av1", ["1.8.0"]),
+                    ("libcurl", ["8"]),
+                    ("liblapack", ["3.9 *netlib"]),
+                    ("libmatio_cpp", ["0.2.3"]),
+                    ("libvips", ["8"]),
+                    ("aws_sdk_cpp", ["1.11.267"]),
+                    ("proj", ["9.3.1"]),
+                    ("libgoogle_cloud_all_devel", ["2.21"]),
+                    ("libgoogle_cloud_aiplatform_devel", ["2.21"]),
+                    ("libpcap", ["1.10"]),
+                    ("pulseaudio_client", ["16.1"]),
+                    ("ffmpeg", ["6"]),
+                    ("graphviz", ["9"]),
+                    ("cudnn", ["8"]),
+                    ("x264", ["1!164.*"]),
+                    ("libcint", ["5.5"]),
+                    ("s2n", ["1.4.4"]),
+                    ("sdl2_ttf", ["2"]),
+                    ("pugixml", ["1.14"]),
+                    ("flatbuffers", ["23.5.26"]),
+                    ("krb5", ["1.20"]),
+                    ("ntl", ["11.4.3"]),
+                    ("libev", ["4.33"]),
+                    ("aws_c_io", ["0.14.4"]),
+                    ("target_gobin", ["${PREFIX}/bin/"]),
+                    ("cxx_compiler", ["gxx"]),
+                    ("ccr", ["1.3"]),
+                    ("postgresql", ["16"]),
+                    ("antic", ["0.2"]),
+                    ("gst_plugins_base", ["1.22"]),
+                    ("rdma_core", ["49"]),
+                    ("libssh2", ["1"]),
+                    ("libtiff", ["4.6"]),
+                    ("kealib", ["1.5"]),
+                    ("mpich", ["4"]),
+                    ("spdlog", ["1.12"]),
+                    ("netcdf_fortran", ["4.6"]),
+                    ("elfutils", ["0.190"]),
+                    ("gsl", ["2.7"]),
+                    ("eclib", ["20231211"]),
+                    ("tiledb", ["2.20"]),
+                    ("libhugetlbfs", ["2"]),
+                    ("nss", ["3"]),
+                    ("netcdf_cxx4", ["4.3"]),
+                    ("pcl", ["1.13.1"]),
+                    ("sox", ["14.4.2"]),
+                    ("superlu_dist", ["8"]),
+                    ("libgoogle_cloud_dialogflow_es_devel", ["2.21"]),
+                    ("attr", ["2.5"]),
+                    ("log4cxx", ["1.2.0"]),
+                    ("aws_c_sdkutils", ["0.1.15"]),
+                    ("libxml2", ["2"]),
+                    ("libsentencepiece", ["0.1.99"]),
+                    ("tbb_devel", ["2021"]),
+                    ("jsoncpp", ["1.9.5"]),
+                    ("msgpack_c", ["6"]),
+                    ("libzip", ["1"]),
+                    ("mkl", ["2023"]),
+                    ("giflib", ["5.2"]),
+                    ("fortran_compiler", ["gfortran"]),
+                    ("libboost_python_devel", ["1.82"]),
+                    ("dbus", ["1"]),
+                    ("libwebp_base", ["1"]),
+                    ("libflint", ["2.9"]),
+                    ("libsoup", ["3"]),
+                    ("libgoogle_cloud_bigtable_devel", ["2.21"]),
+                    ("abseil_cpp", ["20220623.0"]),
+                    ("librdkafka", ["2.2"]),
+                    ("starlink_ast", ["9.2.7"]),
+                    ("lerc", ["4"]),
+                    ("davix", ["0.8"]),
+                    ("libxsmm", ["1"]),
+                    ("poppler", ["23.07"]),
+                    ("dcap", ["2.47"]),
+                    ("gnuradio_core", ["3.10.9"]),
+                    ("gsoap", ["2.8.123"]),
+                    ("fmt", ["10"]),
+                    ("libopencv", ["4.9.0"]),
+                    ("aws_c_http", ["0.8.1"]),
+                    ("lua", ["5"]),
+                    ("qt", ["5.15"]),
+                    ("pcre", ["8"]),
+                    ("ldas_tools_framecpp", ["2.9"]),
+                    ("ruby", ["2.6", "2.5"]),
+                    ("libgoogle_cloud_policytroubleshooter_devel", ["2.21"]),
+                    ("libdap4", ["3.20.6"]),
+                    ("mimalloc", ["2.1.2"]),
+                    ("coin_or_osi", ["0.108"]),
+                    ("gdk_pixbuf", ["2"]),
+                    ("target_goarch", ["amd64"]),
+                    ("assimp", ["5.3.1"]),
+                    ("flann", ["1.9.2"]),
+                    ("gct", ["6.2.1629922860"]),
+                    ("aws_c_auth", ["0.7.16"]),
+                    ("pyqtchart", ["5.15"]),
+                    ("coin_or_clp", ["1.17"]),
+                    ("grpc_cpp", ["1.52"]),
+                    ("libgdal", ["3.8"]),
+                    ("volk", ["3.1"]),
+                    ("curl", ["8"]),
+                    ("geos", ["3.12.1"]),
+                    ("zstd", ["1.5"]),
+                    ("libabseil", ["20240116"]),
+                    ("blas_impl", ["openblas", "mkl", "blis"]),
+                    ("openblas", ["0.3.*"]),
+                    ("zlib_ng", ["2.0"]),
+                    ("liblapacke", ["3.9 *netlib"]),
+                    ("libframel", ["8.41"]),
+                    ("postgresql_plpython", ["16"]),
+                    ("libgoogle_cloud", ["2.21"]),
+                    ("gstreamer", ["1.22"]),
+                    ("lcms", ["2"]),
+                    ("libuuid", ["2"]),
+                    ("coin_or_cgl", ["0.60"]),
+                    ("rust_compiler", ["rust"]),
+                    ("bullet_cpp", ["3.25"]),
+                    ("fftw", ["3"]),
+                    ("tbb", ["2021"]),
+                    ("libsecret", ["0.18"]),
+                    ("libgoogle_cloud_spanner_devel", ["2.21"]),
+                    ("channel_sources", ["conda-forge"]),
+                    ("pcre2", ["10.42"]),
+                    ("glib", ["2"]),
+                    ("google_cloud_cpp_common", ["0.25.0"]),
+                    ("openmpi", ["4"]),
+                    ("ncurses", ["6"]),
+                    ("openh264", ["2.4.1"]),
+                    ("wcslib", ["8"]),
+                    ("soapysdr", ["0.8"]),
+                    ("tinyxml2", ["10"]),
+                    ("libavif", ["1.0.1"]),
+                    ("cairo", ["1"]),
+                    ("icu", ["73"]),
+                    ("libsndfile", ["1.2"]),
+                    ("aom", ["3.7"]),
+                    ("x265", ["3.5"]),
+                    ("jasper", ["4"]),
+                    ("pyqt", ["5.15"]),
+                    ("xrootd", ["5"]),
+                    ("gnutls", ["3.7"]),
+                    ("rocksdb", ["8.0"]),
+                    ("libgit2", ["1.7"]),
+                    ("libgoogle_cloud_oauth2_devel", ["2.21"]),
+                    ("zfp", ["1.0"]),
+                    ("libtorch", ["2.1"]),
+                    ("libarchive", ["3.7"]),
+                    ("freetype", ["2"]),
+                    ("poco", ["1.13.2"]),
+                    ("libthrift", ["0.19.0"]),
+                    ("qt_main", ["5.15"]),
+                    ("libosqp", ["0.6.3"]),
+                    ("ucx", ["1.15.0"]),
+                    ("libgoogle_cloud_automl_devel", ["2.21"]),
+                    ("slepc", ["3.20"]),
+                    ("aws_c_cal", ["0.6.10"]),
+                    ("p11_kit", ["0.24"]),
+                    ("target_platform", ["linux-64"]),
+                    ("libblas", ["3.9 *netlib"]),
+                    ("libgoogle_cloud_bigquery_devel", ["2.21"]),
+                    ("libmicrohttpd", ["1.0"]),
+                    ("mpg123", ["1.32"]),
+                    ("srm_ifce", ["1.24.6"]),
+                    ("petsc", ["3.20"]),
+                    ("libiconv", ["1"]),
+                    ("aws_crt_cpp", ["0.26.2"]),
+                    ("console_bridge", ["1.0"]),
+                    ("libgoogle_cloud_storage_devel", ["2.21"]),
+                    ("lz4_c", ["1.9.3"]),
+                    ("mumps_mpi", ["5.6.2"]),
+                    ("coincbc", ["2.10"]),
+                    ("orc", ["1.9.2"]),
+                    ("libcblas", ["3.9 *netlib"]),
+                    ("readline", ["8"]),
+                    ("nodejs", ["18", "20"]),
+                    ("glpk", ["5.0"]),
+                    ("imath", ["3.1.9"]),
+                    ("gdal", ["3.8"]),
+                    ("nettle", ["3.9"]),
+                    ("qtkeychain", ["0.14"]),
+                    ("c_ares", ["1"]),
+                    ("libduckdb_devel", ["0.9.2"]),
+                    ("occt", ["7.7.2"]),
+                    ("qt6_main", ["6.6"]),
+                    ("perl", ["5.32.1"]),
+                    ("libidn2", ["2"]),
+                    ("pyqtwebengine", ["5.15"]),
+                    ("coin_or_utils", ["2.11"]),
+                    ("libopenvino_dev", ["2023.3.0"]),
+                    ("googleapis_cpp", ["0.10"]),
+                    ("libwebp", ["1"]),
+                    ("coin_or_cbc", ["2.10"]),
+                    ("channel_targets", ["conda-forge main"]),
+                    ("sdl2_image", ["2"]),
+                    ("sdl2_mixer", ["2"]),
+                    ("vtk", ["9.2.6"]),
+                    ("librsvg", ["2"]),
+                    ("jpeg", ["9"]),
+                    ("hdf4", ["4.2.15"]),
+                    ("pytorch", ["2.1"]),
+                    ("libintervalxt", ["3"]),
+                    ("thrift_cpp", ["0.19.0"]),
+                    ("libgoogle_cloud_discoveryengine_devel", ["2.21"]),
+                    ("arpack", ["3.8"]),
+                    ("libtensorflow", ["2.15"]),
+                    ("vlfeat", ["0.9.21"]),
+                    ("snappy", ["1"]),
+                    ("capnproto", ["0.10.2"]),
+                    ("libmatio", ["1.5.26"]),
+                    ("c_compiler", ["gcc"]),
+                    ("cgo_compiler", ["go-cgo"]),
+                    ("ipopt", ["3.14.14"]),
+                    ("libiio", ["0"]),
+                    ("singular", ["4.3.2.p8"]),
+                    ("libhwy", ["1.0"]),
+                    ("zeromq", ["4.3.5"]),
+                    ("pixman", ["0"]),
+                    ("libspatialindex", ["1.9.3"]),
+                    ("libffi", ["3.4"]),
+                    ("nspr", ["4"]),
+                    ("petsc4py", ["3.20"]),
+                    ("libsqlite", ["3"]),
+                    ("pulseaudio", ["16.1"]),
+                    ("libopentelemetry_cpp", ["1.14"]),
+                    ("libptscotch", ["7.0.4"]),
+                    ("libraw", ["0.21"]),
+                    ("libgoogle_cloud_dlp_devel", ["2.21"]),
+                    ("libgoogle_cloud_iam_devel", ["2.21"]),
+                    ("openjpeg", ["2"]),
+                    ("libhwloc", ["2.9.1"]),
+                    ("r_base", ["4.2", "4.3"]),
+                    ("gfal2", ["2.21"]),
+                    ("libexactreal", ["4"]),
+                    ("mkl_devel", ["2023"]),
+                    ("zlib", ["1.2"]),
+                    ("libmed", ["4.1"]),
+                    ("fontconfig", ["2"]),
+                    ("xz", ["5"]),
+                    ("suitesparse", ["5"]),
+                    ("libgoogle_cloud_speech_devel", ["2.21"]),
+                    ("sdl2_net", ["2"]),
+                    ("slepc4py", ["3.20"]),
+                    ("tk", ["8.6"]),
+                    ("libpng", ["1.6"]),
+                    ("libssh", ["0.10"]),
+                    ("urdfdom", ["3.1"]),
+                    ("metis", ["5.1.0"]),
+                    ("libnetcdf", ["4.9.2"]),
+                    ("sdl2", ["2"]),
+                    ("target_goos", ["linux"]),
+                    ("cfitsio", ["4.3.0"]),
+                    ("pulseaudio_daemon", ["16.1"]),
+                    ("mumps_seq", ["5.6.2"]),
+                    ("hdf5", ["1.14.3"]),
+                    ("nccl", ["2"]),
+                    ("libevent", ["2.1.12"]),
+                    ("exiv2", ["0.27"]),
+                    ("libeantic", ["2"]),
+                    ("alsa_lib", ["1.2.10"]),
+                    ("glog", ["0.7"]),
+                    ("libscotch", ["7.0.4"]),
+                    ("cutensor", ["2"]),
+                    ("json_c", ["0.17"]),
+                    ("aws_c_common", ["0.9.13"]),
+                    ("isl", ["0.26"]),
+                    ("openssl", ["3"]),
+                    ("xerces_c", ["3.2"]),
+                    ("cpu_optimization_target", ["nocona"]),
+                    ("libflatsurf", ["3"]),
+                    ("libkml", ["1.3"]),
+                    ("gflags", ["2.2"]),
+                    ("libboost_devel", ["1.82"]),
+                    ("libgoogle_cloud_dialogflow_cx_devel", ["2.21"]),
+                    ("VERBOSE_CM", ["VERBOSE=1"]),
+                    ("openexr", ["3.2"]),
+                    ("scotch", ["7.0.4"]),
+                    ("libgoogle_cloud_pubsub_devel", ["2.21"]),
+                    ("libabseil_static", ["20220623.0"]),
+                    ("go_compiler", ["go-nocgo"]),
+                    ("re2", ["2023.06.02"]),
+                    ("tensorflow", ["2.15"]),
+                    ("libarrow_all", ("12", "14", "13", "15")),
+                    ("arrow_cpp", ("12", "14", "13", "15")),
+                    ("libarrow", ("12", "14", "13", "15")),
+                    ("c_compiler_version", ("10", "12")),
+                    ("fortran_compiler_version", ("10", "12")),
+                    ("cdt_name", ("cos7", "cos6")),
+                    (
+                        "docker_image",
+                        (
+                            "quay.io/condaforge/linux-anvil-cuda:11.2",
+                            "quay.io/condaforge/linux-anvil-cos7-x86_64",
+                        ),
+                    ),
+                    ("cuda_compiler", ("nvcc", "None")),
+                    ("cxx_compiler_version", ("10", "12")),
+                    ("cuda_compiler_version", ("11.2", "None")),
+                    ("libgrpc", ("1.61",)),
+                    ("libprotobuf", ("4.25.2",)),
+                    ("c_stdlib_version", ("2.12",)),
+                    ("c_stdlib", ("sysroot",)),
+                    (
+                        "python",
+                        (
+                            "3.9.* *_cpython",
+                            "3.10.* *_cpython",
+                            "3.11.* *_cpython",
+                            "3.8.* *_cpython",
+                            "3.12.* *_cpython",
+                        ),
+                    ),
+                    ("numpy", ("1.22", "1.22", "1.23", "1.22", "1.26")),
+                    (
+                        "python_impl",
+                        (
+                            "cpython",
+                            "cpython",
+                            "cpython",
+                            "cpython",
+                            "cpython",
+                        ),
+                    ),
+                    (
+                        "zip_keys",
+                        [
+                            ("arrow_cpp", "libarrow", "libarrow_all"),
+                            (
+                                "c_compiler_version",
+                                "cxx_compiler_version",
+                                "fortran_compiler_version",
+                                "cuda_compiler",
+                                "cuda_compiler_version",
+                                "cdt_name",
+                                "docker_image",
+                            ),
+                            ("c_stdlib", "c_stdlib_version"),
+                            ("libgrpc", "libprotobuf"),
+                            ("python", "numpy", "python_impl"),
+                        ],
+                    ),
+                ]
+            ),
+            {
+                "BUILD",
+                "MACOSX_DEPLOYMENT_TARGET",
+                "MACOSX_SDK_VERSION",
+                "build_number_decrement",
+                "c_compiler",
+                "c_compiler_version",
+                "cdt_arch",
+                "cdt_name",
+                "channel_sources",
+                "channel_targets",
+                "cuda_compiler",
+                "cuda_compiler_version",
+                "docker_image",
+                "macos_machine",
+                "macos_min_version",
+                "pin_run_as_build",
+                "target_platform",
+                "zip_keys",
+            },
+            {
+                "c_compiler": ["gcc"],
+                "c_compiler_version": ["10", "12"],
+                "cdt_name": ["cos7", "cos6"],
+                "channel_sources": ["conda-forge"],
+                "channel_targets": ["conda-forge main"],
+                "cuda_compiler": ["nvcc", "None"],
+                "cuda_compiler_version": ["11.2", "None"],
+                "docker_image": [
+                    "quay.io/condaforge/linux-anvil-cuda:11.2",
+                    "quay.io/condaforge/linux-anvil-cos7-x86_64",
+                ],
+                "pin_run_as_build": dict(
+                    [
+                        ("python", {"max_pin": "x.x", "min_pin": "x.x"}),
+                        ("r-base", {"max_pin": "x.x", "min_pin": "x.x"}),
+                        ("flann", {"max_pin": "x.x.x"}),
+                        ("graphviz", {"max_pin": "x"}),
+                        ("libsvm", {"max_pin": "x"}),
+                        ("netcdf-cxx4", {"max_pin": "x.x"}),
+                        ("occt", {"max_pin": "x.x"}),
+                        ("poppler", {"max_pin": "x.x"}),
+                        ("vlfeat", {"max_pin": "x.x.x"}),
+                    ]
+                ),
+                "target_platform": ["linux-64"],
+                "zip_keys": [
+                    ("arrow_cpp", "libarrow", "libarrow_all"),
+                    (
+                        "c_compiler_version",
+                        "cxx_compiler_version",
+                        "fortran_compiler_version",
+                        "cuda_compiler",
+                        "cuda_compiler_version",
+                        "cdt_name",
+                        "docker_image",
+                    ),
+                    ("c_stdlib", "c_stdlib_version"),
+                    ("libgrpc", "libprotobuf"),
+                    ("python", "numpy", "python_impl"),
+                ],
+            },
+        )
+    ],
+)
+def test_get_used_key_values_by_input_order(
+    squished_input_variants,
+    squished_used_variants,
+    all_used_vars,
+    used_key_values,
+):
+    assert (
+        cnfgr_fdstk._get_used_key_values_by_input_order(
+            squished_input_variants,
+            squished_used_variants,
+            all_used_vars,
+        )
+        == used_key_values
+    )