RFC adopt narwhals for dataframe support

At least as of [SLEP018](https://scikit-learn-enhancement-proposals.readthedocs.io/en/latest/slep018/proposal.html), scikit-learn supports dataframes passed as `X`. In #25896 is a further place of current discussions.

This issue is to discuss whether or not, or in which form, a future scikit-learn should depend on [narwhals](https://github.com/narwhals-dev/narwhals) for general dataframe support.

`+` wide df support
`+` less maintenance within scikit-learn
`-` external dependency

@scikit-learn/core-devs @MarcoGorelli

COMMENT:
OK nobody was bold enough to raise concerns yet, so let me the first one to do it 😅.

Being not very familiar with the space, I would need more information to first form an opinion and eventually try to reach a decision on this.

I have to admit my default reaction is conservatism. Adding a new dependency in scikit-learn has always been done with careful consideration about the trade-offs involved.

One of my worry: I have to admit that from afar this ecosystem (to simplify let's say narwhals + polars) moves very fast (which you know can also be a good thing 😉) and there seems to be an impedance mismatch with how fast the core Scientific Python ecosystem is moving.

### What does it bring us?

This has probably been discussed in other issues but it would be nice to have it summarized in this issue where we are trying to make a decision.

- what does depending on narwhals bring us? Is it mostly to do pandas + polars conveniently? Are there any issues with our pandas + polars implementation that depending on narwhals would fix?
- are there many users who care about support for an other dataframe library than pandas or polars?
- What kind of workflows would depending on narwhals unlock for you? What kind of work-around do you currently use?

### How will our code change? aka what does it cost us?

- how much of our code will need to change? Is it mostly a few (5-10) key functions?
- what is your plan for the future? Would the switch to narwhals be done in a single PR? Is it going to be a lengthy per-function/estimator process like array API support (hopefully not 😅)?
- not sure how much work this is, but it would help a lot if your WIP PR https://github.com/scikit-learn/scikit-learn/pull/31127 was changing our code in a way that enables you to do something that wasn't possible before. This would help having some feeling how much change is needed from us to switch to narwhals.

I guess linking to pull requests from other projects that switched to narwhals would be useful as well.

### Discussion of the risks involved

Here are a few things off the top of my head, it would be nice to have these aspects detailed as well in the cost-benefit analysis:
- narwhals stops being maintained and we end up having to maintain a fork. I guess unlikely in the short-term but you know it's hard to make predictions, especially about the future 🔮.
- intersection with array API? narwhals can support Dask for example, and with array-api-extra  array API also supports Dask, how does it interact together in our code base, any caveats there?
- any consideration about the lazy aspects, any insights about supporting it, is it out of scope for now?

Labels: RFC