SLEP006 - Metadata Routing task list

This issue is to track the work we need to do before we can merge `sample-props` branch into `main`:

- [x] Based on this prototype: https://github.com/scikit-learn/scikit-learn/pull/16079
- [x] Merge https://github.com/scikit-learn/scikit-learn/pull/22083 into `sample-props`. This PR only touches `BaseEstimator` and hence _consumers_. It does NOT touch meta-estimators, scorers or cv splitters.
- [x] Work on splitters and merge that into `sample-props`: https://github.com/scikit-learn/scikit-learn/pull/22765
- [x] feature flag: https://github.com/scikit-learn/scikit-learn/issues/26045: https://github.com/scikit-learn/scikit-learn/pull/26103
- [x] Add new terms to the glossary: router, consumer, metadata: https://github.com/scikit-learn/scikit-learn/pull/26685
- [x] Work on scorers and merge that into `sample-props` ~(note that this involves an ongoing discussion on whether we'd like to mutate a scorer or not)~: https://github.com/scikit-learn/scikit-learn/pull/22757
- [x] Get backward compatibility mechanisms in place for meta-estimators which already route given metadata: https://github.com/scikit-learn/scikit-learn/pull/22986
- [ ] Work on meta-estimators: this involves writing `get_metadata_routing` in easy cases, and a whole lot more in cases where we'd like to keep backward compatibility in parsing input args suchs as `estimator__param` in `Pipeline`
- [x] Once all the above is merged into `sample-props`, do a few tests with third party estimators to see if they'd work out of the box. Note that _consumer_ estimators should work out of the box as long as they inherit from `BaseEstimator` and their `fit` accepts metadata as explicit arguments rather than `**kwargs`.
  - Merged w/o completion of meta-estimators, they got done progressively.
- [x] Check whether a library such as cuML could vendor `_metadata_requests.py` and work with scikit-learn meta-estimators w/o depending on the library.
  - dropped the idea of supporting vendoring, they'd need to inherit from `BaseEstimator`
- [x] Refactor tests: https://github.com/scikit-learn/scikit-learn/issues/23918
- [x] Merge `sample-props` into `main`: https://github.com/scikit-learn/scikit-learn/pull/24027

## Enhancements:
- [x] https://github.com/scikit-learn/scikit-learn/pull/24023
- [ ] https://github.com/scikit-learn/scikit-learn/issues/18936
- [x] https://github.com/scikit-learn/scikit-learn/issues/28467

## Cleanup
- [x] https://github.com/scikit-learn/scikit-learn/issues/26505
	- [x] https://github.com/scikit-learn/scikit-learn/pull/26503

## Open issues:
- [x] https://github.com/scikit-learn/scikit-learn/issues/22987
	- [x] https://github.com/scikit-learn/scikit-learn/pull/26506
- [x] https://github.com/scikit-learn/scikit-learn/issues/22988: https://github.com/scikit-learn/scikit-learn/pull/23342
- [ ] https://github.com/scikit-learn/scikit-learn/issues/23920
- [ ] https://github.com/scikit-learn/scikit-learn/issues/23928
	- [ ] global flags: https://github.com/scikit-learn/scikit-learn/issues/26050
	- [ ] auto-routing: https://github.com/scikit-learn/scikit-learn/issues/26179
- [x] https://github.com/scikit-learn/scikit-learn/issues/23933
	- [x] https://github.com/scikit-learn/scikit-learn/pull/26911
- [x] https://github.com/scikit-learn/scikit-learn/issues/25776: https://github.com/scikit-learn/scikit-learn/pull/25852 - resolution is to use the feature flag instead of breaking change.
- [ ] https://github.com/scikit-learn/scikit-learn/issues/27977

Our plan is to hopefully have this feature in 1.1, which we should be releasing in late April/early May.

cc @jnothman @thomasjpfan @lorentzenchr 

## Misc:

- https://github.com/scikit-learn/scikit-learn/pull/25851
- https://github.com/scikit-learn/scikit-learn/pull/27389

-------------------------------------------------------------
From #24027 

Once we're finished with merging PRs to `sample-props` branch, we should merge this.

I've opened this for us to have an idea of what the whole project is touching.

We probably should rebase and merge this instead of squash and merge.

This closes and fixes the following issues and PRs:

- Routing issues:
Enables #6322, 

- AdaBoost:
Fixes #2630, 
Fixes #21706

- ClassifierChain:
Fixes  #11429, 

- FeatureUnion:
Fixes #7136

- LogisticRegressionCV:
Fixes  #8950,

- Multioutput*
Fixes #15953

- OneVsRestClassifier
Fixes #10882

- RFE/RFECV:
Fixes #7308

- *SearchCV:
Fixes #8127, 
Fixes #8158,

- cross_val*:
Fixes #13432, 
Fixes #4632, 
Fixes #7646, 
Fixes #20349

- Stacking*
Fixes #18028,

- Voting*
Fixes #20167

COMMENT:
> Do you see this as a collaborative effort? To what extent can we still use https://github.com/scikit-learn/scikit-learn/pull/20350? To what extent can we share test components? Which of these metaestimators have existing fit param routing that needs to be deprecated?

Most of what's in #20350 can be used. We can open this for collaboration once we figure out a nice way to deprecate existing routing strategies in our meta-estimators. I'll work on it this week.

Labels: Hard, Meta-issue, Metadata Routing