RFC CLI Tool Proposal

## Introduction
Other OSS projects such as `scipy` and `numpy` have made use of development CLI tools such as `runtests.py` and `dev.py` to provide a collected and singular entry point into development tasks for contributors. I find these tools very helpful personally, and think they add a large amount of value, especially for onboarding new contributors. Recently Pandas has also begun discussion and implementation of their own development CLI tool. @noatamir has made a great writeup on some of the benefits and justifications for such a tool (see: https://github.com/pandas-dev/pandas/issues/47700). I highly recommend taking the time to read their writeup. 

Some of the benefits of such a tool include:
1. Increased discoverability of tools, decreasing time spent jumping around documentation. This is especially helpful for those who may not read every single word of documentation (such is my bad habit 😅).
2. Consistency for CI and development. We can establish a 1-1 between CI checks and `dev.py` commands, e.g. `dev.py lint` which could then impose both `black` and `isort` (cf. https://github.com/scikit-learn/scikit-learn/issues/22853)
3. Ease of installation. Users could install whatever dependencies are necessary depending on what workflow they want enabled. A call to `dev.py doc build` could check for the presence of the necessary dependencies and install if needed (or optionally prompt).
4. Reduced barrier to entry. Quite frankly, having to run various commands for tasks, anywhere from linting to building C sources, can be overwhelming, especially to new developers.

Recently `scipy` explored a new solution for developing a unified development CLI tool (cf. https://github.com/scipy/scipy/issues/15489) based on [`do.it`](https://pydoit.org/). Pandas is also exploring a `do.it` based [solution](https://github.com/pandas-dev/pandas/issues/47700). There's been good feedback on this framework and I would propose the same.

## Encompassed Actions
Some actions which would be wrapped by such a development CLI include:
- Dependency installation (could be per-workflow)
- Testing
- Linting
- Extension building
- Documentation building
- Pre-commit installation

## Considerations
As with all things, the question is whether the benefits outweigh the maintenance cost. While projects like scipy with many complex workflows benefit the most from such a tool, personally I think anything to decrease the barrier to entry for scikit-learn development is worth considering.

If we do opt to build such a tool, we can make use of the work already done in scipy as well as the recently released separate [`dev.py` package](https://github.com/scientific-python/dev.py). Either way, the largest modification we'd need to make is alter the `build` command to use `setuptools` instead of `meson`, though that shouldn't be too problematic.

COMMENT:
**edit** I had mis-understood the proposal. Rewriting my comment (and sorry for the mis-understanding, that was a bit dense of me).


# Standardizing our CLIs / shell scripts

I think that the actual proposal above it to standardize the CLIs / shell scripts / Makefiles that we use.

I'm not sure where I sit here. My thoughts:
- Makefile & shell are difficult to maintain and require specific skills that are not share amongst even good Python devs. An argument to get rid of them. Also, they typically do not work on all systems (eg Windows)
- *If it ain't broken don't fix it*. A large reengineering project like this always smells bad. It's likely to consume a lot of energy and be inefficient: in the short term because we won't see gains (gains will be from better maintainability), in the long term because requirements will evolve.

I think that the good solution is to consider case-by-case settings: replacing elements of our toolchain as we go. Ideally the discussion should be case-by-case.

I personally would be fine with having a general policy that we try to prefer Python code to shell. However, what really matters is that the current pool of active devs feel comfortable with changes and are happy about them.

# Making a CLI out of scikit-learn is out of scope.

(not the proposal, but a possible misinterpretation).

We are a Python library. We focus on doing high quality numerical algorithms and the tools around them that are necessary to use them to have valid statistical usage an operationalization.

We should not embark on this. It will be a strong distraction (and we have plenty on our plate to satisfy the above mission), and it will lead to frustration from the users because we will likely develop half-baked answer (difficult to have a proper expressivity in a CLI given the complexity of pipelines + model evaluation).

Finally, such projects can be developed outside scikit-learn. The policy these last years has been to develop new ideas outside, to avoid adding even more weight to an already overburden project.

Labels: RFC, workflow