Stable extender contract via `fit` / `_fit` resp `predict` / `_predict` separation

### Describe the workflow you want to enable

tl;dr, I am suggestion to refactor `scikit-learn` internals to a layer separation with boilerplate between `fit` and `_fit` resp `predict` and `_predict` methods, to make extender interfaces more stable. Also see https://github.com/scikit-learn/scikit-learn/issues/31728

More background: Currently, every time `scikit-learn` releases a new minor version - e.g., 1.5.0, 1.6.0, 1.7.0 - compliant extensions, e.g., custom transformers, classifiers, etc, break - specifically, referring to the API conformance as tested through `check_estimator` or `parametrize_with_checks`.

These repeated breakages in the "extender contract" contrast the stability of the usage contract, which is stable and professionally managed.

For a package like `scikit-learn` which means to be a standard not just for ML algorithms but also an API standard that everyone uses, this is not a good state to be in - "do not break user code" is the maxim that gets broken for power users writing extensions.

Of course maintaining downwards compatibility is not always possible, but nothing should break without a proper warning.

### Describe your proposed solution

The `fit`/`_fit` separation would ensure stability of the extension contract - and would also allow to build secondary deprecation patterns in relation to it.

The (oop) pattern this would implement is the so-called "template pattern".

It would allow to remove likely changing parts such as the boilerplate (e.g., `validate_data` vs `_validate_data` and such) from the extension locus, and thus completely prevent breakage in relation to boilerplate changes.
Reference: https://refactoring.guru/design-patterns/template-method

Examples of how this can be used to improve stability:

* `sktime`, for a different API, has a separation between `fit` calling an internal `_fit`, where change-prone boilerplate is sandwiched between a stable user contract (`fit`) and a stable extender contract (`_fit`); similarly `predict` and `_predict`
* `feature-engine` overrides the `BaseTransformer` `scikit-learn` extension contract with a similar pattern using `super()` calls in `fit` etc.

The proposed `fit`/`_fit` pairing that combines strategy and template pattern can be introduced easily via pure internal refactoring - this also add a layer of tests that checks the stability of the extension contract.

As a side effect, this would likely reduce the amount of copy-pasted boilerplate among the core estimators in `scikit-learn` proper as well.

### Describe alternatives you've considered, if relevant

The obvious alternative is "do nothing" - which will lead to packages with extensions producing their own version of mitigating patterns for the unstable status quo.

The problem with this is that the extending packages are mutually incompatible with their extender pattern. It would be better if there is a single stable extension pattern prescribed - or at least recommended - by `scikit-learn`.

### Additional context

_No response_

COMMENT:
This issue has two aspects to it. One is the separation of fit and _fit, which more generally abides to more advanced object-oriented programming patterns. The other aspect is the ease of implementing scikit-learn compliant packages. That latter issue is a user need, thus a functional request, which is what we should be focusing on.

Regarding using more advanced OOP patterns, there is an explicit choice is scikit-learn to refrain from unneeded sophistication. This choice is partly driven by a Pythonic understanding that object-oriented programming is only one of the paradigms and that lighter ones are preferable when possible. Heavy use of OOP leads to code where the algorithmics are not written in a linear way and grouped in a single place. Given that scikit-learn strives to appeal to contributors with training in statistics or applied maths, being closer to simple procedural code matters particularly. 


Zooming out to the core of the issue: the difficulty to maintain scikit-learn compatible code across different versions. The struggle is real 😀. On the one hand, I know very well, as a developer of packages such as nilearn or skrub, that we do too tight coupling with scikit-learn and as a result updates to scikit-learn break our code. On the other hand, as a scikit-learn developer, I also know that to be able to add more functionality we must change our inner workings, and this is what breaks downstream. Committing to not changing things would freeze the package, making it impossible to implement major additions such as support for new data types. 
But, it is not clear that more object-oriented code is a path to the solution. The problem is too much coupling between code across packages, in particular relying on code that is considered private by scikit-learn developers. OOP is not better than functions in this respect. It may even by considered worse as the API contact of a class is more complex than that of a function. As a consequence, more things can change. 

For the issues that I see, a good set of functions provide the basic building blocks.

Also, it is important to understand that there will be no magic bullet. Importing complex behavior across packages is bound to hit challenges as packages evolve. This is a case for striving to reduce coupling, potentially by, as a downstream package, implementing a bit more code in the downstream code base. 

It would be great if scikit-learn could help more downstream packages, and, as has been mentioned before, we would ideally work on defining better public contracts of utilities to be reused. This is also a lot of work.

Finally, something that is also helpful across the ecosystem is to have continuous integration of downstream packages running the dev version of scikit-learn, to catch difficulties as soon as possible. 


In sum, let's address the functional need, which are about more well-defined and stable tools for implementing scikit-learn compliant models, rather than embark in an OOP rewrite, which will come with its costs and will require an even bigger contract definition and maintenance.

Labels: RFC, Developer API