Repository: pypa/pip

Issue:
`--config-settings` recognizes only the last option
### Description

The doc said:
> The user can supply configuration settings using the `--config-settings` command line option (which can be supplied multiple times, in order to specify multiple settings).

If setting multiple times like this:
```shell
pip wheel --config-settings="--build-option=--cffi" --config-settings="--build-option=--avx2" -v .
```
Only the last option is passed to backend: `{'--build-option': '--avx2'}`

It seems these code let the last option override the previous options:
https://github.com/pypa/pip/blob/c987c682367760415f5ac9afebc0444ef03e8bb4/src/pip/_internal/cli/cmdoptions.py#L803-L806

### Expected behavior

All options are passed to backend: `{'--build-option': ['--cffi', '--avx2']}`

Modify the code like this:
```python
    if dest is None:
        dest = {key: []}
        setattr(parser.values, option.dest, dest)
    dest[key].append(val)
```

### pip version

22.3.1

### Python version

3.10

### OS

Windows 11

### How to Reproduce

1. define a custom PEP-517 hook `get_requires_for_build_wheel`
2. let the hook print the `config_settings` argument

```python
    def get_requires_for_build_wheel(config_settings=None):
        print('config_settings argument:', config_settings)
        return []
````

### Output

_No response_

### Code of Conduct

- [X] I agree to follow the [PSF Code of Conduct](https://www.python.org/psf/conduct/).


Discussion:
@pfmoore the author of the commit https://github.com/pypa/pip/commit/cdeb8f9e63f5881fe2f43d79bba5609848b2f1fa
This command solves the problem.
```shell
pip wheel --config-settings="--build-option=--cffi --avx2" -v .
```
I think there's a genuine bug here.  Does your workaround work for all backends or does it just happen to work for setuptools because it's splitting (`shlex.split`) command-line arguments on its side?  IOW, how does a command-line user construct a [list of config setting values](https://peps.python.org/pep-0517/#config-settings) using pip?
> I think there's a genuine bug here.

Yes, the behavior is different from the document, reopen the issue.

My workaround works for `setuptools` backend.


> IOW, how does a command-line user construct a [list of config setting values](https://peps.python.org/pep-0517/#config-settings) using pip?

Note the following in the `config-settings` documentation:

> Build frontends SHOULD provide some mechanism for users to specify arbitrary string-key/string-value pairs to be placed in this dictionary. 

Specifically, there's no requirement on frontends to provide a mechanism for users to specify anything other than a *string* value.

The pip example in the PEP is just an example of how pip *might* implement `--config-settings`, but it's not actually what we did. I'm not against the idea of allowing something like what is suggested here, but we need to be cautious. It's not pip's responsibility to define the semantics of `config_settings`, that's for backends, so what *should* happen is for backend maintainers to agree on what should be allowed as valid config settings, beyond the minimal "string key, string value" case defined by PEP 517. Once there's an agreed "extended standard", then pip can implement it.

For example, what if a user wants to pass a 1-element list to a backend? How would they do that? We can't mandate that backends have to treat a string value the same as a 1-element list containing that value. When I implemented the feature, I toyed with the idea of allowing JSON-encoded non-string values somehow. But supplying such values on the command line rapidly turns into a quoting nightmare, so I backed off and stuck with the minimum required by the standard.

Setuptools currently accepts lists by splitting. Maybe all backends should do that? I don't know, and it's not really my choice. It would be a lot simpler for frontends if that was the decision, of course.

Having said all of the above, I'm not *necessarily* against someone trying to implement something more complex. But if you do, be aware that any such behaviour will be very much pip-specific, and you can't assume that other frontends (such as `build`, or conda, or whatever) will behave the same, or indeed will go beyond the string value approach covered by the PEP which pip currently uses.
My chief concern is that the homonymous option in build does behave differently.  Somebody coming from pip to build, or from build to pip, as I assume has happened here, will find the discrepancy between the two tools surprising.  I don't have a preference, but I think we shouldn't diverge without good reason.  That could very well mean that build adopts pip's model for parsing config settings.
Good point. IMO, sticking with what's standardised is the correct approach here, which probably means string values only for now. If more complex values are important, then I think the standard needs updating to define the minimal set of types frontends should support. For instance, the build API is a Python API, so there's no reason a backend couldn't choose to interpret strings and numbers differently.
We ran also into this issue in `meson-python` (xref https://github.com/mesonbuild/meson-python/issues/235) and `scipy` (xref https://github.com/scipy/scipy/issues/17736#issuecomment-1373909094). `cibuildwheel` ran into it too (xref https://github.com/pypa/cibuildwheel/issues/1227#issuecomment-1212972828).

`pypa/build` allows using the same key more than once, and merges the result into a list of strings. It would be very useful for `pip` and `pypa/build` to have the same behavior here, and what `pypa/build` does seems reasonable, and I think also matches the intent of PEP 517 (given the example linked above does match).

> Somebody coming from pip to build, or from build to pip, as I assume has happened here, will find the discrepancy between the two tools surprising.

+1 - also timely given the "unification of interfaces would be really nice" discussion happening on Discourse. 

> If more complex values are important, then I think the standard needs updating to define the minimal set of types frontends should support

I don't think there is anything in PEP 517 about this case either way, aside from the example which explicitly shows that `"key1=value1" ... "key1=value2"` gets merged into `"key1" = ["value1", "value2"]` when calling the build backend hook. So I'd suggest following that example. If you want a clarification in writing @pfmoore, where should it go? Writing a whole new PEP just to say "if keys are duplicate, merge them" seems a bit much.
Agreed, I'd be happy with a simple PR to change the spec. Unfortunately, it looks like no-one has migrated PEP 517 to https://packaging.python.org/en/latest/specifications/ yet, so it would have to be a PR changing the PEP itself (which is less than ideal, but what we've done in the past so I'm not bothered by it).

A quick post on discourse, asking if any backend would have a problem with frontends treating duplicated keys as a list of values, should be sufficient to confirm consensus. It might also be worth pointing out that this would mean that backends will never receive a single-element list for an option - instead, that would be passed as a non-list string argument. That's the thing I'm most concerned about (a backend saying "this option is always a list", and not expecting this quirk).

To be clear, I'm not expecting any arguments here, I just want to make sure we don't suddenly start sending unexpected config values to backends.
What do we want to change in the spec?  If it's to provide a schema for config settings, I think that'd be a positive change.  But if it's to define how command-line options in frontends should behave, that seems a little excessive.  It's not unthinkable that another, unaffiliated frontend might wish to read config settings from JSON, like Paul's described above.  Personally, I think it's sufficient for pip and build to decide on a common strategy.  If we're unsure whether build's or pip's approach is better, or if we believe there's room for improvement, we can continue on Discourse.
> What do we want to change in the spec?

I think this (part in italics is added): "Build frontends SHOULD provide some mechanism for users to specify arbitrary string-key/string-value pairs to be placed in this dictionary. _In case a user provides duplicate string-key's, build frontends SHOULD combine the corresponding string-value's into a list of string-values._ For example ..."

> Personally, I think it's sufficient for pip and build to decide on a common strategy.

Either way works for me. It seems like a minimal clarification.
The one thing that still bugs me is that backends will have to special-case a string value as equivalent to a 1-element list containing that value. I assume setuptools does that, as otherwise the proposed new behaviour wouldn't work at all (and from what I understand it does work in build). But it seems like a weird condition to impose on backends, that they have to allow this.

Given that using a space-separated list of values `--config-settings="--build-option=--cffi --avx2" works for setuptools, because the specification of the `--build-option` config setting is that it gets split on spaces, do we actually *need* this change? Are there any backends that require a config option to be supplied as a list? And if so, how do *they* expect a single-item list to be supplied?

In reality, of course, hardly anyone supports config options, and the setuptools `--build-option` case is a holdover from the legacy `setup.py` command line form, so there really isn't any good example of the "right" way to do things here...
> But it seems like a weird condition to impose on backends, that they have to allow this.

Backends do not have to allow this. If they have no use for it, and hence not implement it, then there is also no valid reason for users to provide duplicate keys to the build frontend.

> Are there any backends that require a config option to be supplied as a list? And if so, how do they expect a single-item list to be supplied?

Meson-python for one: https://github.com/mesonbuild/meson-python/issues/235. A single-item list works fine with what both `pip` and `build` do today.

It's possible I think to have a `--config-settings=key="a value"`, so the backend can't be sure about splitting on spaces. Passing multiple `--config-settings` seems more unambiguous. And is already done when keys are not the same, probably for the same reason.
So meson-python also translates singe strings to 1-item lists [here](https://github.com/mesonbuild/meson-python/blob/8a13f8d199e4961fe2b0aeff01d231f510b6c59b/mesonpy/__init__.py#L1086). Cool.

I think the next step is probably for someone to propose a PR to the spec, and a PR for pip.
Meson (not the PEP 517 build backend) definitely supports receiving config values that contain embedded spaces. In fact, it also supports receiving array types like this: `meson setup builddir/ -Dsimpleopt=simpleval "-Doptname=['val1', 'val2', 'val3,has,commas and spaces']"` but if you don't have "unusual" values with commas in them, you can just define it as `meson setup builddir/ -Dsimpleopt=simpleval -Doptname=val1,val2,val3`.

In short, people can put weird things into an option, and having some form of reliably transcribing that is probably useful.

One method is of course --config-settings=json-blob. Another method is having --config-settings=shlex-parsed-string. But they're both quoting hell, just to different degrees, so, I agree -- it's probably best to get out of that game as rapidly as possible and just accumulate opaque values to the right side of an equals sign.
If we’re to change the spec, wouldn’t a simpler solution to make `config_settings` a list of key-value tuples so backends can decide how to parse them?
@uranusjr that does seem like a nicer design, but it runs the risk of backwards compat issues for some backends I think. If they don't support list of string input, they need to be updated first and then pip would only be able to make that change much later. If it's `str | list[str, ...]`, then there's nothing that backends need to do. Do you think it's worth pushing for that larger change?
IMO we can adopt the above discussion:
1,  Only pass the string after the first equal sign to backend.  https://github.com/pypa/pip/issues/11681#issuecomment-1374705381
2, Use this form to deal with multiple specifying: `str | list[str, ...]`. https://github.com/pypa/pip/issues/11681#issuecomment-1374781004

Another question, does the key should be stripped?
If there is a space before the first equal sign, `setuptools` doesn't recognize this key as `--build-option`.
```shell
pip wheel --config-settings="--build-option = --cffi" -v .
```
> If it's `str | list[str, ...]`, then there's nothing that backends need to do

If a user is relying on the current behaviour (a later assignment overwrites the previous value), this would still make the backend crash in a weird way since the backend is only expecting to receive str values. If we want to eliminate this, some sort of flags would be needed in the backend anyway for compatibility. Unless we don’t feel this is worth supporting (by telling users that overwriting values was an accident and considered a bug).
> If a user is relying on the current behaviour (a later assignment overwrites the previous value)

This behavior is different from doc, can be seem as a bug. 

> since the backend is only expecting to receive str values

Backends probably allow the [examples](https://peps.python.org/pep-0517/#config-settings) in PEP-517.

> Unless we don’t feel this is worth supporting (by telling users that overwriting values was an accident and considered a bug)

I don't think it's worth supporting, FWIW.

> A quick post on discourse, asking if any backend would have a problem with frontends treating duplicated keys as a list of values, should be sufficient to confirm consensus.

Done in https://discuss.python.org/t/amending-pep-517-with-a-sentence-on-handling-duplicate-config-settings-keys/23222

Resolving patch:
diff --git a/src/pip/_internal/cli/cmdoptions.py b/src/pip/_internal/cli/cmdoptions.py
--- a/src/pip/_internal/cli/cmdoptions.py
+++ b/src/pip/_internal/cli/cmdoptions.py
@@ -811,7 +811,13 @@ def _handle_config_settings(
     if dest is None:
         dest = {}
         setattr(parser.values, option.dest, dest)
-    dest[key] = val
+    if key in dest:
+        if isinstance(dest[key], list):
+            dest[key].append(val)
+        else:
+            dest[key] = [dest[key], val]
+    else:
+        dest[key] = val
 
 
 config_settings: Callable[..., Option] = partial(


Test patch:
diff --git a/tests/unit/test_pyproject_config.py b/tests/unit/test_pyproject_config.py
--- a/tests/unit/test_pyproject_config.py
+++ b/tests/unit/test_pyproject_config.py
@@ -1,3 +1,5 @@
+from typing import Dict, List
+
 import pytest
 
 from pip._internal.commands import create_command
@@ -36,9 +38,16 @@ def test_set_config_empty_value() -> None:
     assert options.config_settings == {"x": ""}
 
 
-def test_replace_config_value() -> None:
+@pytest.mark.parametrize(
+    ("passed", "expected"),
+    [
+        (["x=hello", "x=world"], {"x": ["hello", "world"]}),
+        (["x=hello", "x=world", "x=other"], {"x": ["hello", "world", "other"]}),
+    ],
+)
+def test_multiple_config_values(passed: List[str], expected: Dict[str, str]) -> None:
     i = create_command("install")
     options, _ = i.parse_args(
-        ["xxx", "--config-settings", "x=hello", "--config-settings", "x=world"]
+        ["xxx", *(f"--config-settings={option}" for option in passed)]
     )
-    assert options.config_settings == {"x": "world"}
+    assert options.config_settings == expected