Stable extender contract via `fit` / `_fit` resp `predict` / `_predict` separation

### Describe the workflow you want to enable

tl;dr, I am suggestion to refactor `scikit-learn` internals to a layer separation with boilerplate between `fit` and `_fit` resp `predict` and `_predict` methods, to make extender interfaces more stable. Also see https://github.com/scikit-learn/scikit-learn/issues/31728

More background: Currently, every time `scikit-learn` releases a new minor version - e.g., 1.5.0, 1.6.0, 1.7.0 - compliant extensions, e.g., custom transformers, classifiers, etc, break - specifically, referring to the API conformance as tested through `check_estimator` or `parametrize_with_checks`.

These repeated breakages in the "extender contract" contrast the stability of the usage contract, which is stable and professionally managed.

For a package like `scikit-learn` which means to be a standard not just for ML algorithms but also an API standard that everyone uses, this is not a good state to be in - "do not break user code" is the maxim that gets broken for power users writing extensions.

Of course maintaining downwards compatibility is not always possible, but nothing should break without a proper warning.

### Describe your proposed solution

The `fit`/`_fit` separation would ensure stability of the extension contract - and would also allow to build secondary deprecation patterns in relation to it.

The (oop) pattern this would implement is the so-called "template pattern".

It would allow to remove likely changing parts such as the boilerplate (e.g., `validate_data` vs `_validate_data` and such) from the extension locus, and thus completely prevent breakage in relation to boilerplate changes.
Reference: https://refactoring.guru/design-patterns/template-method

Examples of how this can be used to improve stability:

* `sktime`, for a different API, has a separation between `fit` calling an internal `_fit`, where change-prone boilerplate is sandwiched between a stable user contract (`fit`) and a stable extender contract (`_fit`); similarly `predict` and `_predict`
* `feature-engine` overrides the `BaseTransformer` `scikit-learn` extension contract with a similar pattern using `super()` calls in `fit` etc.

The proposed `fit`/`_fit` pairing that combines strategy and template pattern can be introduced easily via pure internal refactoring - this also add a layer of tests that checks the stability of the extension contract.

As a side effect, this would likely reduce the amount of copy-pasted boilerplate among the core estimators in `scikit-learn` proper as well.

### Describe alternatives you've considered, if relevant

The obvious alternative is "do nothing" - which will lead to packages with extensions producing their own version of mitigating patterns for the unstable status quo.

The problem with this is that the extending packages are mutually incompatible with their extender pattern. It would be better if there is a single stable extension pattern prescribed - or at least recommended - by `scikit-learn`.

### Additional context

_No response_

COMMENT:
> rather than embark in an OOP rewrite, which will come with its costs and will require an even bigger contract definition and maintenance.

@GaelVaroquaux, I disagree with you on a factual basis. Key points

1. Adopting `fit` / `_fit` makes the extension contract *easier* , not *harder*

* it removes the need to comply with undocumented requirements such as `validate_data`
* historically, it was not possible to comply with the extension contract without using private methods from `scikit-learn`! The `fit` / `_fit` pattern removes that.

2. Adopting `fit` / `_fit` *reduces* code complexity and contract complexity instead of increasing it

* it removes all boilerplate lines from concrete estimators. There will be a large net number of duplicative lines removed from the code base - copied/repeated code is an anti-pattern https://en.wikipedia.org/wiki/Don%27t_repeat_yourself
* the contract becomes simpler to implement and to comply with since less boilerplate is needed in `_fit` than in `fit`

3. Adopting `fit` / `_fit` *reduces* maintenance burden instead of increasing it

* boilerplate code needs to be maintained only in the base class after the rework. Currently, it must be maintained in each and every single estimator within sklearn and across the entire directly dependent ecosystem of extensions. You reduce the burden to constant size, from a burden that scales with the number of estimators in the package and the ecosystem.
* the likelihood and impact of potentially breaking changes gets reduced, many historically breaking changes would not have been breaking (see above)

4. Adopting `fit` / `_fit` makes it *easier*  for your clientele of statisticians and applied math to read and contribute, not harder

* `_fit` and `_predict` now only include the "mathematical core", all the boilerplate is removed - this is clearer for this audience
* this audience also does not want to bother with tags or `validate_data`, see above

Regarding your argumentation, I think it is based on highlighting "OOP" as some kind of dark magic that needs to be avoided. In my opinion, of course overengineering should be avoided, but underengineering can also cause trouble - and imo this is exactly how it looks like! Namely, the significant problems with maintaining estimators and extensions, everyone seems to acknowledges they are there!

In my opinion, the problem can be solved by a bit of pattern awareness in the design. Slapping OOP on everything in an uninformed way is indeed bad and I have seem it done many times (perhaps that is why you are a bit cautious, @GaelVaroquaux) - but this I am pretty sure is a textbook case of the template pattern,

good reference explaining it in a human understandable way:
https://refactoring.guru/design-patterns/template-method

Labels: RFC, Developer API