Calibration cannot handle different dtype for prediction and sample weight.

### Describe the bug

This is from the comment: https://github.com/scikit-learn/scikit-learn/issues/28245#issuecomment-2106845979 . I did not find a corresponding issue. Please close if this is duplicated.


Aligning the types here https://github.com/scikit-learn/scikit-learn/blob/6a2472fa5e48a53907418a427c29562a889bd1a7/sklearn/calibration.py#L842 can help resolve the problem, but the casting is done for every grad calculation. Hopefully there's a better solution.

Users can workaround the issue by using `float32` for sample weights.

### Steps/Code to Reproduce

``` python
import numpy as np
import pandas as pd
import xgboost as xgb
from sklearn.calibration import CalibratedClassifierCV
from sklearn.naive_bayes import GaussianNB

df = pd.DataFrame(
    {"x": np.random.random(size=100), "y": np.random.choice([0, 1], size=100)}
)
sample_weight = np.ones((100))

model = xgb.XGBClassifier()

calibrator = CalibratedClassifierCV(model)

calibrator.fit(df[["x"]], df["y"], sample_weight)
```

### Expected Results

No error.

### Actual Results



Here, xgboost outputs `float32`, but `sample_weight` is `float64`. These mismatched types lead to the following error:

```python-traceback
  File "/home/jiamingy/.anaconda/envs/xgboost_dev/lib/python3.12/site-packages/sklearn/calibration.py", line 673, in _fit_calibrator
    calibrator.fit(this_pred, Y[:, class_idx], sample_weight)
  File "/home/jiamingy/.anaconda/envs/xgboost_dev/lib/python3.12/site-packages/sklearn/calibration.py", line 908, in fit
    self.a_, self.b_ = _sigmoid_calibration(X, y, sample_weight)
                       ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/jiamingy/.anaconda/envs/xgboost_dev/lib/python3.12/site-packages/sklearn/calibration.py", line 855, in _sigmoid_calibration
    opt_result = minimize(
                 ^^^^^^^^^
  File "/home/jiamingy/.anaconda/envs/xgboost_dev/lib/python3.12/site-packages/scipy/optimize/_minimize.py", line 738, in minimize
    res = _minimize_lbfgsb(fun, x0, args, jac, bounds,
          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/jiamingy/.anaconda/envs/xgboost_dev/lib/python3.12/site-packages/scipy/optimize/_lbfgsb_py.py", line 386, in _minimize_lbfgsb
    sf = _prepare_scalar_function(fun, x0, jac=jac, args=args, epsilon=eps,
         ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/jiamingy/.anaconda/envs/xgboost_dev/lib/python3.12/site-packages/scipy/optimize/_optimize.py", line 291, in _prepare_scalar_function
    sf = ScalarFunction(fun, x0, args, grad, hess,
         ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/jiamingy/.anaconda/envs/xgboost_dev/lib/python3.12/site-packages/scipy/optimize/_differentiable_functions.py", line 223, in __init__
    self._update_fun()
  File "/home/jiamingy/.anaconda/envs/xgboost_dev/lib/python3.12/site-packages/scipy/optimize/_differentiable_functions.py", line 295, in _update_fun
    fx = self._wrapped_fun(self.x)
         ^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/jiamingy/.anaconda/envs/xgboost_dev/lib/python3.12/site-packages/scipy/optimize/_differentiable_functions.py", line 21, in wrapped
    fx = fun(np.copy(x), *args)
         ^^^^^^^^^^^^^^^^^^^^^^
  File "/home/jiamingy/.anaconda/envs/xgboost_dev/lib/python3.12/site-packages/scipy/optimize/_optimize.py", line 80, in __call__
    self._compute_if_needed(x, *args)
  File "/home/jiamingy/.anaconda/envs/xgboost_dev/lib/python3.12/site-packages/scipy/optimize/_optimize.py", line 74, in _compute_if_needed
    fg = self.fun(x, *args)
         ^^^^^^^^^^^^^^^^^^
  File "/home/jiamingy/.anaconda/envs/xgboost_dev/lib/python3.12/site-packages/sklearn/calibration.py", line 840, in loss_grad
    l, g = bin_loss.loss_gradient(
           ^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/jiamingy/.anaconda/envs/xgboost_dev/lib/python3.12/site-packages/sklearn/_loss/loss.py", line 258, in loss_gradient
    self.closs.loss_gradient(
  File "sklearn/_loss/_loss.pyx", line 1791, in _loss.CyHalfBinomialLoss.loss_gradient
ValueError: Buffer dtype mismatch, expected 'const float' but got 'double'
```

### Versions

```shell
System:
    python: 3.12.0 | packaged by conda-forge | (main, Oct  3 2023, 08:43:22) [GCC 12.3.0]
executable: /home/jiamingy/.anaconda/envs/xgboost_dev/bin/python3
   machine: Linux-6.8.0-52-generic-x86_64-with-glibc2.39

Python dependencies:
      sklearn: 1.6.1
          pip: 24.3.1
   setuptools: 75.8.0
        numpy: 1.26.4
        scipy: 1.15.1
       Cython: 3.0.11
       pandas: 2.2.3
   matplotlib: 3.10.0
       joblib: 1.4.2
threadpoolctl: 3.5.0

Built with OpenMP: True

threadpoolctl info:
       user_api: blas
   internal_api: mkl
    num_threads: 32
         prefix: libmkl_rt
       filepath: /home/jiamingy/.anaconda/envs/xgboost_dev/lib/libmkl_rt.so.2
        version: 2024.2.2-Product
threading_layer: intel

       user_api: openmp
   internal_api: openmp
    num_threads: 64
         prefix: libomp
       filepath: /home/jiamingy/.anaconda/envs/xgboost_dev/lib/libomp.so
        version: None
```

COMMENT:
Thanks for the report. I think I found the root cause, and I am working on a fix.

Labels: Bug