Repository: pantsbuild/pants

Issue:
Support for pairing specific resolves and interpreter constraints when using `parameterize`
Survey submitter said:

> make it easier (without a macro) to use paramatierze with interpreter_constraints/resolves without generating the full cross product


Discussion:
More generally, I think this request is a request for a way to use `parametrize()` without it doing a Cartesian product of all fields.

So, something like this should have only 3 instances instead of 9 (3 fields * 3 parametrize entries each).

```python
pex_binary(
    ...
    resolve=parametrize("py36", "py37", "py38"),
    interpreter_constraints=parametrize(
        py36=["CPython==3.6.*"],
        py37=["CPython==3.7.*"],
        py38=["CPython==3.8.*"],
    ),
    output_path=parametrize(
        py36="st2-py36.pex",
        py37="st2-py37.pex",
        py38="st2-py38.pex",
    ),
)
```

Maybe this could be done by adding a new special kwarg to parametrize like `cartesian=False` (that seems obtuse - and would probably need a better name). So, something like this:

```python
pex_binary(
    ...
    resolve=parametrize("py36", "py37", "py38", cartesian=False),
    interpreter_constraints=parametrize(
        py36=["CPython==3.6.*"],
        py37=["CPython==3.7.*"],
        py38=["CPython==3.8.*"],
        cartesian=False,
    ),
    output_path=parametrize(
        py36="st2-py36.pex",
        py37="st2-py37.pex",
        py38="st2-py38.pex",
        cartesian=False,
    ),
)
```
Yea. This was discussed in the [original design](https://docs.google.com/document/d/1mzZWnXiE6OkgMH_Dm7WeA3ck9veusQmr_c6GWPb_tsQ/edit#): one other potential syntax for this that was discussed then was:

```python
pex_binary(
    ...
    **parametrize(
      py36=dict(
        resolve="py36",
        interpreter_constraints=["CPython==3.6.*"],
        output_path="st2-py36.pex",
      ),
      py37=dict(
        ..
      },
      py38=dict(
        ..
      ),
    )
)
```

...which I think could be implemented by having `parametrize` always expand to a `dict` containing one well-known member, and then adjusting each syntax to identify that kwarg, and re-apply it appropriately.

The primary question I think is whether it is too magical a syntax (Python users will definitely say "what the heck" when they see it). One way to ameliorate that might be to give the keyword a different name that makes it more obvious what is happening: `parametrize_multiple` or something.
One big change from the original design is that at the time `__defaults__` didn't exist: now that it does, the whole "configuration" concept (which was never implemented) is much less relevant, IMO.
> The primary question I think is whether it is too magical a syntax

Agreed, maybe we could make this syntax work?
```python
@parametrize(
  py36=dict(
    resolve="py36",
    interpreter_constraints=["CPython==3.6.*"],
    output_path="st2-py36.pex",
  ),
  py37=dict(
    ..
  },
  py38=dict(
    ..
  ),
)
pex_binary(
    ...
)
```

> > The primary question I think is whether it is too magical a syntax
> 
> Agreed, maybe we could make this syntax work?
> ```python
> @parametrize(
>   py36=dict(
>     resolve="py36",
>     interpreter_constraints=["CPython==3.6.*"],
>     output_path="st2-py36.pex",
>   ),
>   py37=dict(
>     ..
>   },
>   py38=dict(
>     ..
>   ),
> )
> pex_binary(
>     ...
> )
> ```
> 

This is not valid python (you can only decorate class or function definitions). But we could use a `with` block:

```python
with parametrize(
    py36=dict(
        resolve="py36",
        interpreter_constraints=["CPython==3.6.*"],
        output_path="st2-py36.pex",
    ),
    py37=dict(
        ...
    },
    py38=dict(
        ...
    ),
):
    pex_binary(
        ...
    )
```
Or maybe allow parametrize to take a target as an arg:

```python
parametrize(
    # arg for the target (but only one)
    pex_binary(
        ...
    ),
    # kwargs for the target field dicts (like overriddes)
    py36=dict(
        resolve="py36",
        interpreter_constraints=["CPython==3.6.*"],
        output_path="st2-py36.pex",
    ),
    py37=dict(
        ...
    },
    py38=dict(
        ...
    ),
)
```

I think this one is my favorite interface so far.
> This is not valid python

Doh. of course.


What I like with the context approach, is that it could be used ~sensibly also with `__defaults__`:

```python

with parametrize(...):
    # Parametrized defaults for all `pex_binary` targets in this subtree. 
    __defaults__(pex_binary=dict(...))

```

Another idea is to support specifying groups of parametrized fields that should go together as one:
```python
pex_binary(
    ...
    resolve=parametrize("py36", "py37", "py38"),
    interpreter_constraints=parametrize(
        py36=["CPython==3.6.*"],
        py37=["CPython==3.7.*"],
        py38=["CPython==3.8.*"],
    ),
    output_path=parametrize(
        py36="st2-py36.pex",
        py37="st2-py37.pex",
        py38="st2-py38.pex",
    ),
    group_parametrized_fields=[("resolve", "interpreter_constraints", "output_path")],
)
```

Or, as inline options (which is pretty close to @cognifloyd's `cartesian=False` option but supports fields that should not take part in this grouping):

```python
pex_binary(
    ...
    resolve=parametrize("py36", "py37", "py38", parametrize_group="py-version"),
    interpreter_constraints=parametrize(
        py36=["CPython==3.6.*"],
        py37=["CPython==3.7.*"],
        py38=["CPython==3.8.*"],
        parametrize_group="py-version",
    ),
    output_path=parametrize(
        py36="st2-py36.pex",
        py37="st2-py37.pex",
        py38="st2-py38.pex",
        parametrize_group="py-version",
    ),
)
```

I don't like the idea of injecting a `group_parametrized_fields` field to every target.

I like `parametrize_group` as a kwarg to `parametrize`. I think that's the cleanest UX suggestion so far. Using the term `cartesian` would be awful ux imo, so this is much better.
Given build files are Python, another option might be allowing using a normal loop:

```python
py_versions = [("py36", "CPython==3.6.*"), ...]
for short, ic in py_versions:
    pex_binary(
        name=placeholder_name("whatever", short=short, ic=ic),
        resolve=short,
        interpreter_constraints=ic,
        output_path="st2-{short}.pex"
    )
```

Where `placeholder_name` is a new (as yet unnamed) construct for explicitly computing the appropriate name/address as `parametrize` would. Potentially this might even be unnecessary, if there's no functionality built off the `name@k=v` constructed address, e.g. a user can just compute an appropriate name themselves (for that specific example, `name=f"whatever-{short}"` is probably good). If there is special functionality related to parametrized addresses, then it might be.

This does make the build file config less declarative, though.

Any extra non-grouped parametrizations can be nested loops, or just `parametrize` calls within the target.

> Or, as inline options (which is pretty close to cognifloyd's cartesian=False option but supports fields that should not take part in this grouping):

I wonder if that might be error-prone and/or limiting:

- I'd be slightly concerned about making a typo like `parametrize_group="py-versoin"` in one of the groups and accidentally ending up with a multiplication.
- It seems annoying to list the paired options far apart from each other.
- If parametrising by a non-string values (or values that aren't simple strings), passing those values as kwargs to a later `parametrize` call seems difficult?

Thoughts?
The loop construct works today, except for the `placeholder_name` obviously. I'm not aware of anything relying on the parametrized naming of things so this could well serve as a work-around for that--but I see it as such, a work around.

I agree with the raised points about the ergonomics of the previous idea and agree it would be better to allow providing the grouped parametrized values together as one unit. (so, maybe something like the "parametrize-as-context-manager" approach or along those lines).

@cognifloyd 
> I don't like the idea of injecting a `group_parametrized_fields` field to every target.

To be clear, I think this would be a construct on the `TargetAdaptor` and not be registered as a target field in the normal sense at all. With that said, it would likely be preferable with a solution that didn't have to touch on the low level adaptors at all.
with all of this, I think the original one presented by @stuhood hits the closest, maybe even support using the target type for each group to make it perhaps clearer what is going on:

```python
pex_binary(
    ...
    **parametrize_group(
      py36=pex_binary(
        resolve="py36",
        interpreter_constraints=["CPython==3.6.*"],
        output_path="st2-py36.pex",
      ),
      py37=pex_binary(
        resolve="py37",
        interpreter_constraints=["CPython==3.7.*"],
        output_path="st2-py37.pex",
      },
      py38=pex_binary(
        resolve="py38",
        interpreter_constraints=["CPython==3.8.*"],
        output_path="st2-py38.pex",
      ),
    )
)
```

And thinking about how this could be implemented, I think this would just be syntactic sugar for something that you could do directly with just a regular target and some options to each `parametrize`d field value, like the `parametrize_group=` but now with the benefit of not having to worry about typos etc. i.e. the above could be the equiv of:

```python
pex_binary(
    ...
    resolve=parametrize(py36="py36", py37="py37", py38="py38", parametrize_group=1111),
    interpreter_constraints=parametrize(
        py36=["CPython==3.6.*"],
        py37=["CPython==3.7.*"],
        py38=["CPython==3.8.*"],
        parametrize_group=1111,
    ),
    output_path=parametrize(
        py36="st2-py36.pex",
        py37="st2-py37.pex",
        py38="st2-py38.pex",
        parametrize_group=1111,
    ),
)
```

----

This should be able to support multiple templates too:

```python
target(
  ...,
  **parametrize_group(a=target(f1=12, f2=98), b=target(f1=34, f2=76)),
  **parametrize_group(c=target(f3="q", f4="w"), d=target(f3="e", f4="r")),
)
```

sugar for:

```python
target(
  ...,
  f1=parametrize(a=12, b=34, parametrize_group=1111),
  f2=parametrize(a=98, b=76, parametrize_group=1111),
  f3=parametrize(c="q", d="e", parametrize_group=2222),
  f4=parametrize(c="w, d="r", parametrize_group=2222),
)
```

resulting in these expanded targets:

```python
target(..., f1=12, f2=98, f3="q", f4="w")
target(..., f1=12, f2=98, f3="e", f4="r")
target(..., f1=34, f2=76, f3="q", f4="w")
target(..., f1=34, f2=76, f3="e", f4="r")
```

> Given build files are Python, another option might be allowing using a normal loop:

That is effectively what I'm doing today:
```python
for ic_name, ic in supported_python_interpreter_constraints().items():
    pex_binary(
        name=f"st2-{ic_name}.pex",
        output_path=f"st2-{ic_name}.pex",
        interpreter_constraints=ic,
        dependencies=[
            # this should depend on all python_distribution targets
            ...
        ],
        include_tools=True,  # include pex.tools to populate a venv from the pex
        include_sources=False,  # always includes generated wheels, so transitive sources not needed
        venv_hermetic_scripts=False,  # do not add -sE to script shebangs
    )
```

And I have `supported_python_interpreter_constraints()` defined as a macro:
```python
# use this to parametrize targets as necessary.
# eg: interpreter_constraints=parametrize(**supported_python_interpreter_constraints())
def supported_python_interpreter_constraints():
    return dict(
        py36=["CPython==3.6.*"],
        py37=["CPython==3.7.*"],
        py38=["CPython==3.8.*"],
    )
```

---------------------------

The `parametrize_group` thing looks great @kaos. Using `**parametrize_group(` instead of just `**parametrize(` is excellent because it highlights that "this is related to, but not the same as, `parametrize`". So, I vote for the suggested UX in https://github.com/pantsbuild/pants/issues/18292#issuecomment-1523406501
throwing some more ideas on this bonfire:
```python
pex_binary(
    parametrize("py36",
      resolve="py36",
      interpreter_constraints=["CPython==3.6.*"],
      output_path="st2-py36.pex",
    ),
    parametrize("py37",
      resolve="py37",
      interpreter_constraints=["CPython==3.7.*"],
      output_path="st2-py37.pex",
    ),
    parametrize("py38",
      resolve="py38",
      interpreter_constraints=["CPython==3.8.*"],
      output_path="st2-py38.pex",
    ),
    ...,
)
```

That is, you can provide a `parametrize` object as a positional arg to the target, with the first value being the parametriziation "name", and the kwargs are field values for that group.


Resolving patch:
diff --git a/src/python/pants/engine/internals/defaults.py b/src/python/pants/engine/internals/defaults.py
--- a/src/python/pants/engine/internals/defaults.py
+++ b/src/python/pants/engine/internals/defaults.py
@@ -51,7 +51,7 @@ def create(
         return cls(
             *map(freeze, parametrize.args),
             **{kw: freeze(arg) for kw, arg in parametrize.kwargs.items()},
-        )
+        ).to_weak()
 
 
 @dataclass
@@ -94,10 +94,30 @@ def get_frozen_defaults(self) -> BuildFileDefaults:
             {
                 target_alias: FrozenDict(
                     {
-                        field_type.alias: self._freeze_field_value(field_type, default)
-                        for field_alias, default in fields.items()
-                        for field_type in self._target_type_field_types(types[target_alias])
-                        if field_alias in (field_type.alias, field_type.deprecated_alias)
+                        **{
+                            field_type.alias: self._freeze_field_value(field_type, default)
+                            for field_alias, default in fields.items()
+                            for field_type in self._target_type_field_types(types[target_alias])
+                            if field_alias in (field_type.alias, field_type.deprecated_alias)
+                        },
+                        **{
+                            key: ParametrizeDefault(
+                                parametrize.group_name,
+                                **{
+                                    field_type.alias: self._freeze_field_value(field_type, default)
+                                    for field_alias, default in parametrize.kwargs.items()
+                                    for field_type in self._target_type_field_types(
+                                        types[target_alias]
+                                    )
+                                    if field_alias
+                                    in (field_type.alias, field_type.deprecated_alias)
+                                },
+                            )
+                            .to_weak()
+                            .to_group()
+                            for key, parametrize in fields.items()
+                            if isinstance(parametrize, Parametrize) and parametrize.is_group
+                        },
                     }
                 )
                 for target_alias, fields in self.defaults.items()
@@ -190,15 +210,24 @@ def _process_defaults(
                     ).keys()
                 )
 
-                for field_alias in default.keys():
-                    if field_alias not in valid_field_aliases:
-                        if ignore_unknown_fields:
-                            del raw_values[field_alias]
-                        else:
-                            raise InvalidFieldException(
-                                f"Unrecognized field `{field_alias}` for target {target_type.alias}. "
-                                f"Valid fields are: {', '.join(sorted(valid_field_aliases))}.",
-                            )
+                def _check_field_alias(field_alias: str) -> None:
+                    if field_alias in valid_field_aliases:
+                        return
+                    if not ignore_unknown_fields:
+                        raise InvalidFieldException(
+                            f"Unrecognized field `{field_alias}` for target {target_type.alias}. "
+                            f"Valid fields are: {', '.join(sorted(valid_field_aliases))}.",
+                        )
+                    elif field_alias in raw_values:
+                        del raw_values[field_alias]
+
+                for field_alias, field_value in default.items():
+                    if isinstance(field_value, Parametrize) and field_value.is_group:
+                        field_value.to_weak()
+                        for parametrize_field_alias in field_value.kwargs.keys():
+                            _check_field_alias(parametrize_field_alias)
+                    else:
+                        _check_field_alias(field_alias)
 
                 # Merge all provided defaults for this call.
                 defaults.setdefault(target_type.alias, {}).update(raw_values)
diff --git a/src/python/pants/engine/internals/parametrize.py b/src/python/pants/engine/internals/parametrize.py
--- a/src/python/pants/engine/internals/parametrize.py
+++ b/src/python/pants/engine/internals/parametrize.py
@@ -5,8 +5,14 @@
 
 import dataclasses
 import itertools
+import operator
+from collections import defaultdict
 from dataclasses import dataclass
-from typing import Any, Iterator, cast
+from enum import Enum
+from functools import reduce
+from typing import Any, Iterator, Mapping, cast
+
+from typing_extensions import Self
 
 from pants.build_graph.address import BANNED_CHARS_IN_PARAMETERS
 from pants.engine.addresses import Address
@@ -20,7 +26,7 @@
     TargetTypesToGenerateTargetsRequests,
 )
 from pants.util.frozendict import FrozenDict
-from pants.util.strutil import bullet_list, softwrap
+from pants.util.strutil import bullet_list, pluralize, softwrap
 
 
 def _named_args_explanation(arg: str) -> str:
@@ -38,12 +44,39 @@ class Parametrize:
     means that individual Field instances need not be aware of it.
     """
 
+    class _MergeBehaviour(Enum):
+        # Do not merge this parametrization.
+        never = "never"
+        # Discard this parametrization with `other`.
+        replace = "replace"
+
     args: tuple[str, ...]
     kwargs: FrozenDict[str, ImmutableValue]
+    is_group: bool
+    merge_behaviour: _MergeBehaviour = dataclasses.field(compare=False)
 
     def __init__(self, *args: str, **kwargs: Any) -> None:
         object.__setattr__(self, "args", args)
         object.__setattr__(self, "kwargs", FrozenDict.deep_freeze(kwargs))
+        object.__setattr__(self, "is_group", False)
+        object.__setattr__(self, "merge_behaviour", Parametrize._MergeBehaviour.never)
+
+    def keys(self) -> tuple[str]:
+        return (f"parametrize_{hash(self.args)}:{id(self)}",)
+
+    def __getitem__(self, key) -> Any:
+        if isinstance(key, str) and key.startswith("parametrize_"):
+            return self.to_group()
+        else:
+            raise KeyError(key)
+
+    def to_group(self) -> Self:
+        object.__setattr__(self, "is_group", True)
+        return self
+
+    def to_weak(self) -> Self:
+        object.__setattr__(self, "merge_behaviour", Parametrize._MergeBehaviour.replace)
+        return self
 
     def to_parameters(self) -> dict[str, Any]:
         """Validates and returns a mapping from aliases to parameter values.
@@ -74,6 +107,37 @@ def to_parameters(self) -> dict[str, Any]:
             parameters[arg] = arg
         return parameters
 
+    @property
+    def group_name(self) -> str:
+        assert self.is_group
+        if len(self.args) == 1:
+            name = self.args[0]
+            banned_chars = BANNED_CHARS_IN_PARAMETERS & set(name)
+            if banned_chars:
+                raise Exception(
+                    f"In {self}:\n  Parametrization group name `{name}` contained separator characters "
+                    f"(`{'`,`'.join(banned_chars)}`)."
+                )
+            return name
+        else:
+            raise ValueError(
+                softwrap(
+                    f"""
+                    A parametrize group must begin with the group name followed by the target field
+                    values for the group.
+
+                    Example:
+
+                        target(
+                            ...,
+                            **parametrize("group-a", field_a=1, field_b=True),
+                        )
+
+                    Got: `{self!r}`
+                    """
+                )
+            )
+
     @classmethod
     def expand(
         cls, address: Address, fields: dict[str, Any | Parametrize]
@@ -85,16 +149,26 @@ def expand(
         separate calls.
         """
         try:
+            parametrizations = cls._collect_parametrizations(fields)
+            cls._check_parametrizations(parametrizations)
             parametrized: list[list[tuple[str, str, Any]]] = [
                 [
                     (field_name, alias, field_value)
                     for alias, field_value in v.to_parameters().items()
                 ]
-                for field_name, v in fields.items()
-                if isinstance(v, Parametrize)
+                for field_name, v in parametrizations.get(None, ())
+            ]
+            parametrized_groups: list[tuple[str, str, Parametrize]] = [
+                ("parametrize", group_name, vs[0][1])
+                for group_name, vs in parametrizations.items()
+                if group_name is not None
             ]
         except Exception as e:
-            raise Exception(f"Failed to parametrize `{address}`:\n{e}") from e
+            raise Exception(f"Failed to parametrize `{address}`:\n  {e}") from e
+
+        if parametrized_groups:
+            # Add the groups as one vector for the cross-product.
+            parametrized.append(parametrized_groups)
 
         if not parametrized:
             yield (address, fields)
@@ -111,14 +185,83 @@ def expand(
                 {field_name: alias for field_name, alias, _ in parametrized_args}
             )
             parametrized_args_fields = tuple(
-                (field_name, field_value) for field_name, _, field_value in parametrized_args
+                (field_name, field_value)
+                for field_name, _, field_value in parametrized_args
+                # Exclude any parametrize group
+                if not (isinstance(field_value, Parametrize) and field_value.is_group)
             )
             expanded_fields: dict[str, Any] = dict(non_parametrized + parametrized_args_fields)
+            expanded_fields.update(
+                # There will be at most one group per cross product.
+                next(
+                    (
+                        field_value.kwargs
+                        for _, _, field_value in parametrized_args
+                        if isinstance(field_value, Parametrize) and field_value.is_group
+                    ),
+                    {},
+                )
+            )
             yield expanded_address, expanded_fields
 
+    @staticmethod
+    def _collect_parametrizations(
+        fields: dict[str, Any | Parametrize]
+    ) -> Mapping[str | None, list[tuple[str, Parametrize]]]:
+        parametrizations = defaultdict(list)
+        for field_name, v in fields.items():
+            if not isinstance(v, Parametrize):
+                continue
+            group_name = None if not v.is_group else v.group_name
+            parametrizations[group_name].append((field_name, v))
+        return parametrizations
+
+    @staticmethod
+    def _check_parametrizations(
+        parametrizations: Mapping[str | None, list[tuple[str, Parametrize]]]
+    ) -> None:
+        for group_name, groups in parametrizations.items():
+            if group_name is not None and len(groups) > 1:
+                group = Parametrize._combine(*(group for _, group in groups))
+                groups.clear()
+                groups.append(("combined", group))
+
+        parametrize_field_names = {field_name for field_name, v in parametrizations.get(None, ())}
+        parametrize_field_names_from_groups = {
+            field_name
+            for group_name, groups in parametrizations.items()
+            if group_name is not None
+            for field_name in groups[0][1].kwargs.keys()
+        }
+        conflicting = parametrize_field_names.intersection(parametrize_field_names_from_groups)
+        if conflicting:
+            raise ValueError(
+                softwrap(
+                    f"""
+                    Conflicting parametrizations for {pluralize(len(conflicting), "field", include_count=False)}:
+                    {', '.join(sorted(conflicting))}
+                    """
+                )
+            )
+
+    @staticmethod
+    def _combine(head: Parametrize, *tail: Parametrize) -> Parametrize:
+        return reduce(operator.add, tail, head)
+
+    def __add__(self, other: Parametrize) -> Parametrize:
+        if not isinstance(other, Parametrize):
+            raise TypeError(f"Can not combine {self} with {other!r}")
+        if self.merge_behaviour is Parametrize._MergeBehaviour.replace:
+            return other
+        if other.merge_behaviour is Parametrize._MergeBehaviour.replace:
+            return self
+        if self.is_group and other.is_group:
+            raise ValueError(f"Parametrization group name is not unique: {self.group_name!r}")
+        raise ValueError(f"Can not combine parametrizations: {self} | {other}")
+
     def __repr__(self) -> str:
-        strs = [str(s) for s in self.args]
-        strs.extend(f"{alias}={value}" for alias, value in self.kwargs.items())
+        strs = [repr(s) for s in self.args]
+        strs.extend(f"{alias}={value!r}" for alias, value in self.kwargs.items())
         return f"parametrize({', '.join(strs)})"
 
 


Test patch:
diff --git a/src/python/pants/engine/internals/build_files_test.py b/src/python/pants/engine/internals/build_files_test.py
--- a/src/python/pants/engine/internals/build_files_test.py
+++ b/src/python/pants/engine/internals/build_files_test.py
@@ -6,7 +6,7 @@
 import logging
 import re
 from textwrap import dedent
-from typing import cast
+from typing import Any, Mapping, cast
 
 import pytest
 
@@ -543,6 +543,70 @@ def test_parametrize_defaults(target_adaptor_rule_runner: RuleRunner) -> None:
     assert target_adaptor.kwargs["tags"] == ParametrizeDefault(a=("a", "root"), b=("non-root", "b"))
 
 
+def test_parametrized_groups(target_adaptor_rule_runner: RuleRunner) -> None:
+    def _determenistic_parametrize_group_keys(value: Mapping[str, Any]) -> dict[str, Any]:
+        # The `parametrize` object uses a unique generated field name when splatted onto a target
+        # (in order to provide a helpful error message in case of non-unique group names), but the
+        # part up until `:` is determenistic on the group name, which we need to exploit in the
+        # tests using parametrize groups.
+        return {key.rsplit(":", 1)[0]: val for key, val in value.items()}
+
+    target_adaptor_rule_runner.write_files(
+        {
+            "hello/BUILD": dedent(
+                """\
+                mock_tgt(
+                  description="desc for a and b",
+                  **parametrize("a", tags=["opt-a"], resolve="lock-a"),
+                  **parametrize("b", tags=["opt-b"], resolve="lock-b"),
+                )
+                """
+            ),
+        }
+    )
+
+    target_adaptor = target_adaptor_rule_runner.request(
+        TargetAdaptor,
+        [TargetAdaptorRequest(Address("hello"), description_of_origin="tests")],
+    )
+    assert _determenistic_parametrize_group_keys(
+        target_adaptor.kwargs
+    ) == _determenistic_parametrize_group_keys(
+        dict(
+            description="desc for a and b",
+            **Parametrize("a", tags=["opt-a"], resolve="lock-a"),  # type: ignore[arg-type]
+            **Parametrize("b", tags=["opt-b"], resolve="lock-b"),
+        )
+    )
+
+
+def test_default_parametrized_groups(target_adaptor_rule_runner: RuleRunner) -> None:
+    target_adaptor_rule_runner.write_files(
+        {
+            "hello/BUILD": dedent(
+                """\
+                __defaults__({mock_tgt: dict(**parametrize("a", tags=["from default"]))})
+                mock_tgt(
+                  tags=["from target"],
+                  **parametrize("a"),
+                  **parametrize("b", tags=["from b"]),
+                )
+                """
+            ),
+        }
+    )
+    address = Address("hello")
+    target_adaptor = target_adaptor_rule_runner.request(
+        TargetAdaptor,
+        [TargetAdaptorRequest(address, description_of_origin="tests")],
+    )
+    targets = tuple(Parametrize.expand(address, target_adaptor.kwargs))
+    assert targets == (
+        (address.parametrize(dict(parametrize="a")), dict(tags=["from target"])),
+        (address.parametrize(dict(parametrize="b")), dict(tags=("from b",))),
+    )
+
+
 def test_augment_target_field_defaults(target_adaptor_rule_runner: RuleRunner) -> None:
     target_adaptor_rule_runner.write_files(
         {
diff --git a/src/python/pants/engine/internals/defaults_test.py b/src/python/pants/engine/internals/defaults_test.py
--- a/src/python/pants/engine/internals/defaults_test.py
+++ b/src/python/pants/engine/internals/defaults_test.py
@@ -4,6 +4,7 @@
 from __future__ import annotations
 
 from collections import namedtuple
+from typing import Any, Mapping
 
 import pytest
 
@@ -87,6 +88,14 @@ def test_assumptions(
     )
 
 
+def _determenistic_parametrize_group_keys(value: Mapping[str, Any]) -> dict[str, Any]:
+    # The `parametrize` object uses a unique generated field name when splatted onto a target (in
+    # order to provide a helpful error message in case of non-unique group names), but the part up
+    # until `:` is determenistic on the group name, which we need to exploit in the tests using
+    # parametrize groups.
+    return {key.rsplit(":", 1)[0]: val for key, val in value.items()}
+
+
 Scenario = namedtuple(
     "Scenario",
     "path, args, kwargs, expected_defaults, expected_error, parent_defaults",
@@ -235,15 +244,49 @@ def test_assumptions(
         ),
         pytest.param(
             Scenario(
-                args=({Test1Target.alias: {"tags": Parametrize(["foo"], ["bar"], baz=["baz"])}},),
+                args=(
+                    {
+                        Test1Target.alias: {
+                            "tags": Parametrize(["foo"], ["bar"], baz=["baz"]),
+                            **Parametrize(
+                                "splat", description="splat-desc", dependencies=["splat:dep"]
+                            ),
+                        }
+                    },
+                ),
                 expected_defaults={
-                    "test_type_1": {
-                        "tags": ParametrizeDefault(("foo",), ("bar",), baz=("baz",)),  # type: ignore[arg-type]
-                    }
+                    "test_type_1": _determenistic_parametrize_group_keys(
+                        {
+                            "tags": ParametrizeDefault(("foo",), ("bar",), baz=("baz",)),  # type: ignore[arg-type]
+                            **ParametrizeDefault(
+                                "splat", description="splat-desc", dependencies=["splat:dep"]
+                            ),
+                        }
+                    )
                 },
             ),
             id="parametrize default field value",
         ),
+        pytest.param(
+            Scenario(
+                args=(
+                    {
+                        Test1Target.alias: {
+                            **Parametrize("splat", description="splat-desc", whats_this="now"),
+                        }
+                    },
+                ),
+                kwargs=dict(ignore_unknown_fields=True),
+                expected_defaults={
+                    "test_type_1": _determenistic_parametrize_group_keys(
+                        {
+                            **ParametrizeDefault("splat", description="splat-desc")  # type: ignore[list-item]
+                        }
+                    )
+                },
+            ),
+            id="parametrize ignore unknown fields",
+        ),
         pytest.param(
             Scenario(
                 args=(
@@ -281,6 +324,7 @@ def test_set_defaults(
         )
         defaults.set_defaults(*scenario.args, **scenario.kwargs)
         actual_defaults = {
-            tgt: dict(field_values) for tgt, field_values in defaults.get_frozen_defaults().items()
+            tgt: _determenistic_parametrize_group_keys(field_values)
+            for tgt, field_values in defaults.get_frozen_defaults().items()
         }
         assert scenario.expected_defaults == actual_defaults
diff --git a/src/python/pants/engine/internals/parametrize_test.py b/src/python/pants/engine/internals/parametrize_test.py
--- a/src/python/pants/engine/internals/parametrize_test.py
+++ b/src/python/pants/engine/internals/parametrize_test.py
@@ -52,6 +52,19 @@ def test_to_parameters_failure(exception_str: str, args: list[Any], kwargs: dict
     assert exception_str in str(exc.value)
 
 
+@pytest.mark.parametrize(
+    "exception_str,args,kwargs",
+    [
+        ("A parametrize group must begin with the group name", [], {}),
+        ("group name `bladder:dust` contained separator characters (`:`).", ["bladder:dust"], {}),
+    ],
+)
+def test_bad_group_name(exception_str: str, args: list[Any], kwargs: dict[str, Any]) -> None:
+    with pytest.raises(Exception) as exc:
+        Parametrize(*args, **kwargs).to_group().group_name
+    assert exception_str in str(exc.value)
+
+
 @pytest.mark.parametrize(
     "expected,fields",
     [
@@ -73,17 +86,77 @@ def test_to_parameters_failure(exception_str: str, args: list[Any], kwargs: dict
             ],
             {"f": Parametrize("b", "c"), "x": Parametrize("d", "e")},
         ),
+        (
+            [
+                ("a@parametrize=A", {"f0": "c", "f1": 1, "f2": 2}),
+                ("a@parametrize=B", {"f0": "c", "f1": 3, "f2": 4}),
+            ],
+            {
+                "f0": "c",
+                **Parametrize("A", f1=1, f2=2),
+                **Parametrize("B", f1=3, f2=4),
+            },
+        ),
+        (
+            [
+                ("a@parametrize=A", {"f": 1}),
+                ("a@parametrize=B", {"f": 2}),
+                ("a@parametrize=C", {"f": "x", "g": ()}),
+            ],
+            # Using a dict constructor rather than a dict literal to get the same semantics as when
+            # we declare a target in a BUILD file.
+            dict(
+                # Field overridden by some parametrize groups.
+                f="x",
+                **Parametrize("A", f=1),  # type: ignore[arg-type]
+                **Parametrize("B", f=2),
+                **Parametrize("C", g=[]),
+            ),
+        ),
     ],
 )
 def test_expand(
     expected: list[tuple[str, dict[str, Any]]], fields: dict[str, Any | Parametrize]
 ) -> None:
     assert sorted(expected) == sorted(
-        (address.spec, result_fields)
-        for address, result_fields in Parametrize.expand(Address("a"), fields)
+        (
+            (address.spec, result_fields)
+            for address, result_fields in Parametrize.expand(Address("a"), fields)
+        ),
+        key=lambda value: value[0],
     )
 
 
+@pytest.mark.parametrize(
+    "fields, expected_error",
+    [
+        (
+            dict(
+                f=Parametrize("x", "y"),
+                g=Parametrize("x", "y"),
+                h=Parametrize("x", "y"),
+                x=5,
+                z=6,
+                **Parametrize("A", f=1),  # type: ignore[arg-type]
+                **Parametrize("B", g=2, x=3),
+            ),
+            "Failed to parametrize `a:a`:\n  Conflicting parametrizations for fields: f, g",
+        ),
+        (
+            dict(
+                f="x",
+                **Parametrize("A", a=1, b=3),  # type: ignore[arg-type]
+                **Parametrize("A", a=2, c=4),
+            ),
+            "Failed to parametrize `a:a`:\n  Parametrization group name is not unique: 'A'",
+        ),
+    ],
+)
+def test_expand_error_cases(fields: dict[str, Any], expected_error: str) -> None:
+    with pytest.raises(Exception, match=expected_error):
+        _ = list(Parametrize.expand(Address("a"), fields))
+
+
 def test_get_superset_targets() -> None:
     def tgt(addr: Address) -> GenericTarget:
         return GenericTarget({}, addr)