Automatically move `y` (and `sample_weight`) to the same device and namespace as `X`

(From https://github.com/scikit-learn/scikit-learn/pull/27800#issuecomment-1878709518 by @ogrisel)

The proposal/idea is to allow `y` to not be on the same device (and namespace?) as `X` when using Array API inputs. Currently we require/assume that they are on the same device and namespace, it is a requirement. However pipelines can't modify `y` which means it is not possible to move from CPU to GPU as one of the steps of the pipeline, the whole pipeline has to stay on one device. The below details some example and code to motivate allowing `X` ad `y` being on different devices (and namespaces).

---

Suppose we have:

```python
>>> import torch
>>> from sklearn import set_config
>>> from sklearn.datasets import make_regression
>>> from sklearn.linear_model import Ridge
>>> set_config(array_api_dispatch=True)
>>> X, y = make_regression(n_samples=int(1e5), n_features=int(1e3), random_state=0)
>>> X_torch_cuda = torch.tensor(X).to("cuda")
>>> y_torch_cuda = torch.tensor(y).to("cuda")
```

I did a quick benchmark with timeit on a host with a 32 cores CPU and an A100 GPU: we get a bit more than 10x speed-up (which is in the range of what I would have expected):

```python
>>> %time Ridge(solver="svd").fit(X, y)
CPU times: user 1min 29s, sys: 1min 4s, total: 2min 34s
Wall time: 6.18 s
Ridge(solver='svd')
>>> %time Ridge(solver="svd").fit(X_torch_cuda, y_torch_cuda)
CPU times: user 398 ms, sys: 2.74 ms, total: 401 ms
Wall time: 402 ms
Ridge(solver='svd')
```

I also tried the following:

```python
>>> Ridge(solver="svd").fit(X_torch_cuda, y)
Traceback (most recent call last):
  Cell In[36], line 1
    Ridge(solver="svd").fit(X_torch_cuda, y)
  File ~/code/scikit-learn/sklearn/base.py:1194 in wrapper
    return fit_method(estimator, *args, **kwargs)
  File ~/code/scikit-learn/sklearn/linear_model/_ridge.py:1197 in fit
    device_ = device(*input_arrays)
  File ~/code/scikit-learn/sklearn/utils/_array_api.py:104 in device
    raise ValueError("Input arrays use different devices.")
ValueError: Input arrays use different devices.
```

I think it might be reasonable to expect this pattern to fail in general. explicitly ask the the user to provide inputs with consistently allocated data buffers.

However, we might want to be more lenient for the particular case of `y` (and `sample_weight`) and change `Ridge.fit` to automatically move `y` to the same namespace and device as `X`:

```python
y = xp.asarray(y, device=device(X))
```

The reason would be to improve the usability of the Array API for the following pipelines:


```python
X_pandas_df, y_pandas_series = fetch_some_pandas_data()

pipeline = make_pipeline(
    some_column_transformer(),  # works on CPU on the input dataframe
    FunctionTransformer(func=lambda X: torch.tensor(X).to("float32").to("cuda")),
    Ridge(solver="svd"),
)
pipeline.fit(X_pandas_df, y_pandas_series)
```

The pipeline steps can only transform `X` and not `y` (or `sample_weight`). So it means that user would have to instead call:

```python
pipeline.fit(X_pandas_df, torch.tensor(y_pandas_series).to("cuda"))
```

which feels a bit weird/cumbersome to me.

This might not be a big deal though and I don't want to delay this PR to get a consensus on this particular point: I think it's fine the way it is for now but we might want come back to the UX of pipelines with Array API steps later. I will open a dedicated issue.

---

If we take action there are a few estimators that need double checking. For example `LinearDiscriminantAnalysis` currently raises low level PyTorch exceptions when `X` and `y` are on different devices.

COMMENT:
Hm, I think these are separate issues. `y` and `sample_weight` follows `X` and `y_true` follows `y_pred` are principles already resolved and nicely documented via the new PR #31452.

But apart from when we had defined: When users pass arrays from different namespaces into the same pipeline (more generally speaking, not only pipeline object), they cannot expect it to just work. But I would see that as a separate problem to solve.

The latter question is how to handle cases when we have not explicitely defined that one a certain param follows another param (such as all arrays that are passed into init or fit follow `X` or `y_true` follows `y_pred`. And I would suggest not to deal with this at all _here_(!) and simply document that we are not dealing with it in the docs for now.

On the long run, we could make up a new error class for these kind of things and raise this. I think that would be cool and it could be a new cross-the-board-issue that we could implement also in places where it hadn't been implemented before.

But really, that does not interfere with the problem we're solving here in this topic and it doesn't block any PR, because it is an additional thing we could do as a service to the users.

Labels: Array API