Repository: pypa/setuptools

Issue:
[FR] User configurable get_requires_for_build_wheel and get_requires_for_build_sdist
### What's the problem this feature will solve?

https://github.com/pypa/setuptools/issues/2823 proposes to remove the legacy `setup_requires` option. Currently, `setup_requires` can be set statically (which has been replaced by the `requires` key in `pyproject.toml`), or dynamically during the running of the `setup.py` file. This latter method of dynamically setting the build requirements can be replaced more cleanly by the two optional PEP 517 hooks `get_requires_for_build_wheel` and `get_requires_for_build_sdist`. Currently setuptools does not allow users to override and/or set the output of these hooks.

### Describe the solution you'd like

I'm not proposing a specific solution (I suspect the best solution will be the one that best matches what users need, and aligns with the internal design of setuptools), but here are three possible designs:

1. Add two new arguments to `setuptools.setup`: `sdist_requires` and `wheel_requires`. These would be lists of requirements, similar to `install_requirements`. This would likely be the least change to existing setup.py files, but what that implies for the internal flow of setuptools is unclear. Additionally, this does not pass `config_settings` to the user code, which could be desired by users. It would be assumed that the presence of `sdist_requires` and `wheel_requires` would override `setup_requires` whilst that option was allowed.

2. Like 1, but instead of the options being lists of requirements, instead a function matching the interface of the PEP 517 hooks would be given. This does require more changes to user code, but means that the user code can handle `config_settings`. I'm not sure there's much difference between 1 and 2 from a setuptools internal-design perspective though (as both are being handled completely within the `setup.py` module).

3. Add `sdist_requires` and `wheel_requires` to `setup.cfg`. The values would follow the same format as `build-backend` from PEP 517, with the callables being pointed to matching the interface of the relevant hook. Depending on the internals of setuptools, this could allow setuptools to only run the `setup.py` file on the mandatory hooks.

### Alternative Solutions

_No response_

### Additional context

This came from https://github.com/pypa/setuptools/issues/2823, from what I can see there's not been a discussion around this.

I'm also not aware of build backends using this in anger, last time I surveyed the ecosystem none of the backends provided a way to set this, so I wouldn't be surprised if this causes some breakage in some frontends (even if it is part of PEP 517).

### Code of Conduct

- [X] I agree to follow the PSF Code of Conduct


Discussion:
Thanks for filing this. Agreed it's something worth exploring. I'm especially interested in providing an interface to provide build-only requirements (not needed for sdist or metadata generation), as I'm aware of one case where users would like to include numpy during builds but don't want to pay the cost of installing numpy during metadata generation.

Can you provide an example of a problem you're trying to solve (or an example project where static build requirements is insufficient)? I acknowledge that PEP 518 build requirements are less flexible than `setup_requires`, but it's the dynamism of setuptools that the packaging ecosystem is attempting to avoid. I'd like to see a compelling use-case that would show the value of dynamic requirements before considering supporting them in this new packaging world.

The downside of dynamic requirements is that a downstream consumer can't inspect in advance what requirements might be needed, so it adds complexity to the build and limits the viable environments in which it can be installed.

As an extreme example, imagine a project that uses `random.choice([build-dep1, build-dep2])` for its build dependencies. It would be near impossible for a downstream consumer to support that package in an offline installer; it couldn't determine which dependencies need to be present to build/install the package.

It's already hard enough to support the environment markers (a build on macOS might by satisfied by a different set of requirements than on Windows), but at least that system is well-defined and backed by supporting libraries. If backends start allowing arbitrary logic for specifying requirements, it becomes dramatically more difficult for vendors to work with the tools.
I 100% agree with not making things worse for downstream users, I'm aiming to make things better by providing the build metadata via standard mechanisms, rather than having to build things in odd ways.

My example use-case is being able to choose between an [MPI](https://en.wikipedia.org/wiki/Message_Passing_Interface) and non-MPI build of [h5py](https://www.h5py.org/) (with the non-MPI wheel being the default, and provided as a wheel on PyPI). Currently users can set `HDF5_MPI="ON"` which modifies `setup_requires` to include mpi4py, and tells the Cython code to enable the MPI features. This makes it much easier for users to build from source (which has to be done for MPI—there not a single MPI library, rather multiple different implementations and versions, and usually users don't get a choice of which one to use, it comes as part of the cluster), so keeping equivalent functionality would be nice.

I don't know of other libraries which have a similar use-case (I think you would need a library using C-extensions which in turn depends on a library with C-extensions *and* the first library cares about how the latter was configured), but I may be self-selecting out which libraries I use.

Some things that would be really nice, which are beyond the scope of this issue (as they probably require a PEP or updating existing PEPs) would be to store the versions used to build the sdist/wheel in the sdists/wheels metadata, and to pass the versions of the build dependencies installed to sdist/wheel builder (this could use `config_settings`). These would hopefully make it easier to manage the dynamism of `get_requires_for_build_*`.

One alternative solution to the example use-case above which I've just though of (which could go under alternative solutions, happy to copy it there if you want me to), is telling users to use non-python package managers such as [spack](https://spack.readthedocs.io/en/latest/index.html), but I suspect that's not going to go over well with some users (given the pip vs. conda arguments...).
Hi @aragilar, have you considered adding a thin layer on top of `setuptools.build_meta` as defined in https://www.python.org/dev/peps/pep-0517/#in-tree-build-backends?

For example, you could have a `_build_backend.py` file:

```python
from setuptools import build_meta as _orig

prepare_metadata_for_build_wheel = _orig.prepare_metadata_for_build_wheel
build_wheel = _orig.build_wheel
build_sdist = _orig.build_sdist


def get_requires_for_build_wheel(self, config_settings=None):
    return _orig.get_requires_for_build_wheel(config_settings) + [...]


def get_requires_for_build_sdist(self, config_settings=None):
    return _orig.get_requires_for_build_sdist(config_settings) + [...]
```

and then tell the front-end to use it via `pyproject.toml`

```toml
[build-system]
requires = ["setuptools", "wheel"]
build-backend = "_build_backend"
backend-path = ["."]
```
(Please don't put `.` on the path, create a dedicated folder.)

Resolving patch:
diff --git a/docs/conf.py b/docs/conf.py
--- a/docs/conf.py
+++ b/docs/conf.py
@@ -95,6 +95,7 @@
 github_sponsors_url = f'{github_url}/sponsors'
 extlinks = {
     'user': (f'{github_sponsors_url}/%s', '@'),  # noqa: WPS323
+    'pypi': ('https://pypi.org/project/%s', '%s'),
 }
 extensions += ['sphinx.ext.extlinks']
 


Test patch: