Talip7/scikit-learn-issues-multilabel huggingface dataset:
Add BlockingTimeSeriesSplit

<img width="612" height="352" alt="Image" src="https://github.com/user-attachments/assets/1d0bdf46-31c6-4531-83d3-ddf33618b5b2" />

COMMENT:
Sorry but right now it looks like you haven't done the minimum amount of work for this issue to be useful.

Please add more details, in particular:
- why would it be useful for you? Describe your use case in a few sentences.
- what are the limitations of the current time-series splitters in scikit-learn?

If I don't hear back in a week or so, I'll close this.

Labels: Needs Info, Closing candidate
⚠️ CI failed on Linux_Runs.pylatest_conda_forge_mkl (last failure: Nov 01, 2025) ⚠️

**CI failed on [Linux_Runs.pylatest_conda_forge_mkl](https://dev.azure.com/scikit-learn/scikit-learn/_build/results?buildId=81941&view=logs&j=dde5042c-7464-5d47-9507-31bdd2ee0a3a)** (Nov 01, 2025)
- Test Collection Failure

COMMENT:
I would like to solve this issue

Labels: Needs Triage
⚠️ CI failed on Linux_Runs.pylatest_conda_forge_mkl (last failure: Nov 01, 2025) ⚠️

**CI failed on [Linux_Runs.pylatest_conda_forge_mkl](https://dev.azure.com/scikit-learn/scikit-learn/_build/results?buildId=81941&view=logs&j=dde5042c-7464-5d47-9507-31bdd2ee0a3a)** (Nov 01, 2025)
- Test Collection Failure

COMMENT:
## CI is no longer failing! ✅

[Successful run](https://dev.azure.com/scikit-learn/scikit-learn/_build/results?buildId=81955&view=logs&j=dde5042c-7464-5d47-9507-31bdd2ee0a3a) on Nov 02, 2025

Labels: Needs Triage
⚠️ CI failed on Linux_Runs.pylatest_conda_forge_mkl (last failure: Nov 01, 2025) ⚠️

**CI failed on [Linux_Runs.pylatest_conda_forge_mkl](https://dev.azure.com/scikit-learn/scikit-learn/_build/results?buildId=81941&view=logs&j=dde5042c-7464-5d47-9507-31bdd2ee0a3a)** (Nov 01, 2025)
- Test Collection Failure

COMMENT:
It's a HTTP 403 in `fetch_olivetti_faces` see #30761 for more details. I am going to close this one, since for now it doesn't happen often enough to do something about it.

Labels: Needs Triage
ENH: Add overlay_dx_score metric for regression evaluation

### Describe the workflow you want to enable

## Current Problem

When evaluating regression models, especially for time series forecasting, existing metrics (MAE, RMSE, R²) don't capture how well predictions align visually with actual values across different tolerance levels.

## Desired Workflow
```python
from sklearn.metrics import overlay_dx_score
from sklearn.model_selection import cross_val_score

# Simple evaluation
score = overlay_dx_score(y_true, y_pred)

# Cross-validation with custom scorer
from sklearn.metrics import make_scorer
scorer = make_scorer(overlay_dx_score)
scores = cross_val_score(model, X, y, scoring=scorer, cv=5)

# GridSearchCV
from sklearn.model_selection import GridSearchCV
grid = GridSearchCV(model, param_grid, scoring=scorer)
grid.fit(X, y)
```

## Use Cases

1. Time series forecasting evaluation where visual alignment matters
2. Scenarios where outlier sensitivity of MSE/RMSE is problematic
3. Understanding prediction quality at multiple tolerance levels
4. Model selection when alignment is more important than absolute error


## Proposed Implementation

Add `overlay_dx_score()` function to `sklearn.metrics`.

### Function Signature
```python
def overlay_dx_score(y_true, y_pred, *,
                     sample_weight=None,
                     multioutput='uniform_average',
                     max_percentage=100.0,
                     min_percentage=0.1,
                     step=0.1):
    """
    Overlay DX metric for evaluating regression predictions.
    
    Measures alignment by computing area under curve showing percentage 
    of predictions within tolerance intervals of varying sizes.
    
    Returns float between 0 and 1, higher is better.
    """
```

### How It Works

1. Define tolerance intervals as percentages of the data range
2. For each tolerance level, compute percentage of predictions within tolerance
3. Calculate area under the resulting curve
4. Normalize by maximum possible area

### Mathematical Formula
```
overlay_dx = A / A_max
```

where:
- A = area under the overlay percentage curve
- A_max = maximum possible area (max_percentage × 100)

### Key Features

- Supports `sample_weight` (sklearn standard)
- Supports `multioutput` regression
- Less sensitive to outliers than MSE/RMSE
- Provides interpretable score (0 to 1 range)

### Implementation Details

- Pure NumPy implementation, no external dependencies
- Vectorized operations for efficiency
- Follows sklearn API conventions
- Complete numpydoc documentation
- 95%+ test coverage ready

## References

- Repository: https://github.com/Smile-SA/overlay_dx
- Currently used internally for time series forecasting evaluation
- Compatible implementation exists: works with cross_val_score, GridSearchCV, Pipeline

### Describe alternatives you've considered, if relevant

_No response_

### Additional context

_No response_

COMMENT:
I am working on it.

Labels: New Feature, Needs Info
StackingRegressor is incompatible with TimeSeriesSplit as cv parameter, raising "ValueError: cross_val_predict only works for partitions"

https://github.com/scikit-learn/scikit-learn/blob/0f9b6a6c0bbbcbbb7f509016fa1006facfe1ba8d/sklearn/model_selection/_validation.py#L1185

### `scikit-learn` version and operating system

scikit-learn version: 1.5.2 (also affects 1.4.x)
Operating System: (e.g., macOS, Windows, Linux)
Python version: (e.g., 3.10)

### The bug

When using `StackingRegressor` (or `StackingClassifier`), passing a `TimeSeriesSplit` object to the `cv` parameter fails, raising a `ValueError: cross_val_predict only works for partitions`.

This check is fundamentally incompatible with `TimeSeriesSplit`, which correctly reserves the initial chunk of data for training and never uses it as a test set.

This bug makes it impossible to use `StackingRegressor` for time-series forecasting without resorting to a leaky workaround like `KFold(shuffle=False)`.

### Steps/Code to Reproduce

Here is a minimal reproducible example:

```python
import numpy as np
import sklearn
from sklearn.model_selection import TimeSeriesSplit, KFold
from sklearn.ensemble import StackingRegressor
from sklearn.linear_model import LinearRegression

print(f"scikit-learn version: {sklearn.__version__}")

# 1. Create dummy time-series data
X = np.random.rand(100, 2)
y = np.random.rand(100)

# 2. Define estimators
estimators = [
    ('lr1', LinearRegression()),
    ('lr2', LinearRegression())
]

# 3. Define the TimeSeriesSplit (the correct tool for this)
# n_splits=5 will reserve the first 1/6th of data for training only.
tscv = TimeSeriesSplit(n_splits=5)

# 4. Create the Stacker using TimeSeriesSplit
stacker_tscv = StackingRegressor(
    estimators=estimators,
    final_estimator=LinearRegression(),
    cv=tscv  # Pass the TimeSeriesSplit
)

# 5. This will FAIL
try:
    print("\nAttempting to fit with TimeSeriesSplit...")
    stacker_tscv.fit(X, y)
    print("SUCCESS: TimeSeriesSplit worked.")
except ValueError as e:
    print(f"FAILED with TimeSeriesSplit: {e}")

# 6. Show the "workaround" that succeeds (but leaks data)
print("\nAttempting to fit with KFold(shuffle=False)...")
kf_cv = KFold(n_splits=5, shuffle=False)
stacker_kf = StackingRegressor(
    estimators=estimators,
    final_estimator=LinearRegression(),
    cv=kf_cv
)
try:
    stacker_kf.fit(X, y)
    print("SUCCESS: KFold(shuffle=False) worked.")
except Exception as e:
    print(f"FAILED with KFold(shuffle=False): {e}")
```

### Impact 
This bug effectively breaks **StackingRegressor** for all time-series applications. It forces users to choose between:

Downgrading to scikit-learn==1.3.2 (where this check did not exist?). Not entirely sure which version it got introduced! 

Using KFold(shuffle=False) as the cv parameter, which introduces severe data leakage and leads to invalid models.

COMMENT:
Yeah StackingRegressor does indeed require a partition-inducing CV strategy. However, the issue is that StackingRegressor won't let us use TimeSeriesSplit. KFold for example is fine, but that's not useful for time series forecasting.

Labels: Needs Info
StackingRegressor is incompatible with TimeSeriesSplit as cv parameter, raising "ValueError: cross_val_predict only works for partitions"

https://github.com/scikit-learn/scikit-learn/blob/0f9b6a6c0bbbcbbb7f509016fa1006facfe1ba8d/sklearn/model_selection/_validation.py#L1185

### `scikit-learn` version and operating system

scikit-learn version: 1.5.2 (also affects 1.4.x)
Operating System: (e.g., macOS, Windows, Linux)
Python version: (e.g., 3.10)

### The bug

When using `StackingRegressor` (or `StackingClassifier`), passing a `TimeSeriesSplit` object to the `cv` parameter fails, raising a `ValueError: cross_val_predict only works for partitions`.

This check is fundamentally incompatible with `TimeSeriesSplit`, which correctly reserves the initial chunk of data for training and never uses it as a test set.

This bug makes it impossible to use `StackingRegressor` for time-series forecasting without resorting to a leaky workaround like `KFold(shuffle=False)`.

### Steps/Code to Reproduce

Here is a minimal reproducible example:

```python
import numpy as np
import sklearn
from sklearn.model_selection import TimeSeriesSplit, KFold
from sklearn.ensemble import StackingRegressor
from sklearn.linear_model import LinearRegression

print(f"scikit-learn version: {sklearn.__version__}")

# 1. Create dummy time-series data
X = np.random.rand(100, 2)
y = np.random.rand(100)

# 2. Define estimators
estimators = [
    ('lr1', LinearRegression()),
    ('lr2', LinearRegression())
]

# 3. Define the TimeSeriesSplit (the correct tool for this)
# n_splits=5 will reserve the first 1/6th of data for training only.
tscv = TimeSeriesSplit(n_splits=5)

# 4. Create the Stacker using TimeSeriesSplit
stacker_tscv = StackingRegressor(
    estimators=estimators,
    final_estimator=LinearRegression(),
    cv=tscv  # Pass the TimeSeriesSplit
)

# 5. This will FAIL
try:
    print("\nAttempting to fit with TimeSeriesSplit...")
    stacker_tscv.fit(X, y)
    print("SUCCESS: TimeSeriesSplit worked.")
except ValueError as e:
    print(f"FAILED with TimeSeriesSplit: {e}")

# 6. Show the "workaround" that succeeds (but leaks data)
print("\nAttempting to fit with KFold(shuffle=False)...")
kf_cv = KFold(n_splits=5, shuffle=False)
stacker_kf = StackingRegressor(
    estimators=estimators,
    final_estimator=LinearRegression(),
    cv=kf_cv
)
try:
    stacker_kf.fit(X, y)
    print("SUCCESS: KFold(shuffle=False) worked.")
except Exception as e:
    print(f"FAILED with KFold(shuffle=False): {e}")
```

### Impact 
This bug effectively breaks **StackingRegressor** for all time-series applications. It forces users to choose between:

Downgrading to scikit-learn==1.3.2 (where this check did not exist?). Not entirely sure which version it got introduced! 

Using KFold(shuffle=False) as the cv parameter, which introduces severe data leakage and leads to invalid models.

COMMENT:
I'm wondering if the fix would be modifying **StackingRegressor** to implement within .fit a custom cross_val_predict only if something like TimeSeriesSplit is chosen 🤔

Labels: Needs Info
Array API support for LogisticRegression with LBFGS

Even if `scipy.optimize` does not support array API yet, we can probably work around it to perform the computation of the gradient using the input data array namespace and convert the gradient values to numpy prior to feeding them back to the solver at each iteration.

Assuming `n_samples >> n_features`, this is likely to be a worthwhile way to get a large speed-up on GPU devices for that estimator despite back and forth conversion to and from device for the parameters array.

To compute the gradient we perform `X @ coef + intercept` which leads to "raw predictions". Those raw predictions are fed along with `y` to the Cython loss. This will be an `(n_samples, n_classes)` array. For a first iteration, we can convert that to NumPy and continue using Cython. Alternatively, we can write pure Python array API capable version of the gradient method of the loss functions. We should start with log loss for which we can already refactor and move the `sklearn.utils._array_api._half_multinomial_loss` function (introduced for metrics) into the `sklearn._loss` package to add its gradient/Hessian counterparts.

We can leave the other losses as raising `NotImplementError` for now.

COMMENT:
For a start maybe I can try adding the `gradient` part of `HalfMultinomialLoss` as a dedicated PR? Then we can add array API support for `LogisticRegression` with `lbfgs` in another PR. What do you think?
But I think we would also need `HalfBinomialLoss` refactored with respect to the array API.

Labels: High Priority, Array API
Array API support for LogisticRegression with LBFGS

Even if `scipy.optimize` does not support array API yet, we can probably work around it to perform the computation of the gradient using the input data array namespace and convert the gradient values to numpy prior to feeding them back to the solver at each iteration.

Assuming `n_samples >> n_features`, this is likely to be a worthwhile way to get a large speed-up on GPU devices for that estimator despite back and forth conversion to and from device for the parameters array.

To compute the gradient we perform `X @ coef + intercept` which leads to "raw predictions". Those raw predictions are fed along with `y` to the Cython loss. This will be an `(n_samples, n_classes)` array. For a first iteration, we can convert that to NumPy and continue using Cython. Alternatively, we can write pure Python array API capable version of the gradient method of the loss functions. We should start with log loss for which we can already refactor and move the `sklearn.utils._array_api._half_multinomial_loss` function (introduced for metrics) into the `sklearn._loss` package to add its gradient/Hessian counterparts.

We can leave the other losses as raising `NotImplementError` for now.

COMMENT:
I find it weird to merge a PR that implements some private functions/methods that are never called in any public counterparts. I would rather introduce such new functions/methods only when actually needed. Otherwise we might not realize that they have a design defect of some kind.

Labels: High Priority, Array API
Array API support for LogisticRegression with LBFGS

Even if `scipy.optimize` does not support array API yet, we can probably work around it to perform the computation of the gradient using the input data array namespace and convert the gradient values to numpy prior to feeding them back to the solver at each iteration.

Assuming `n_samples >> n_features`, this is likely to be a worthwhile way to get a large speed-up on GPU devices for that estimator despite back and forth conversion to and from device for the parameters array.

To compute the gradient we perform `X @ coef + intercept` which leads to "raw predictions". Those raw predictions are fed along with `y` to the Cython loss. This will be an `(n_samples, n_classes)` array. For a first iteration, we can convert that to NumPy and continue using Cython. Alternatively, we can write pure Python array API capable version of the gradient method of the loss functions. We should start with log loss for which we can already refactor and move the `sklearn.utils._array_api._half_multinomial_loss` function (introduced for metrics) into the `sklearn._loss` package to add its gradient/Hessian counterparts.

We can leave the other losses as raising `NotImplementError` for now.

COMMENT:
I understand and agree.

Labels: High Priority, Array API
Array API support for LogisticRegression with LBFGS

Even if `scipy.optimize` does not support array API yet, we can probably work around it to perform the computation of the gradient using the input data array namespace and convert the gradient values to numpy prior to feeding them back to the solver at each iteration.

Assuming `n_samples >> n_features`, this is likely to be a worthwhile way to get a large speed-up on GPU devices for that estimator despite back and forth conversion to and from device for the parameters array.

To compute the gradient we perform `X @ coef + intercept` which leads to "raw predictions". Those raw predictions are fed along with `y` to the Cython loss. This will be an `(n_samples, n_classes)` array. For a first iteration, we can convert that to NumPy and continue using Cython. Alternatively, we can write pure Python array API capable version of the gradient method of the loss functions. We should start with log loss for which we can already refactor and move the `sklearn.utils._array_api._half_multinomial_loss` function (introduced for metrics) into the `sklearn._loss` package to add its gradient/Hessian counterparts.

We can leave the other losses as raising `NotImplementError` for now.

COMMENT:
@ogrisel How about first adding array API support for `LogisticRegression` with **LBFGS** without modifying the `loss` or `gradient` and simply `converting to numpy` when calling the optimization related code. Then maybe later we can see if we want to refactor the `loss` and `gradient` as well. This might help in working on this systematically.

Labels: High Priority, Array API
Array API support for LogisticRegression with LBFGS

Even if `scipy.optimize` does not support array API yet, we can probably work around it to perform the computation of the gradient using the input data array namespace and convert the gradient values to numpy prior to feeding them back to the solver at each iteration.

Assuming `n_samples >> n_features`, this is likely to be a worthwhile way to get a large speed-up on GPU devices for that estimator despite back and forth conversion to and from device for the parameters array.

To compute the gradient we perform `X @ coef + intercept` which leads to "raw predictions". Those raw predictions are fed along with `y` to the Cython loss. This will be an `(n_samples, n_classes)` array. For a first iteration, we can convert that to NumPy and continue using Cython. Alternatively, we can write pure Python array API capable version of the gradient method of the loss functions. We should start with log loss for which we can already refactor and move the `sklearn.utils._array_api._half_multinomial_loss` function (introduced for metrics) into the `sklearn._loss` package to add its gradient/Hessian counterparts.

We can leave the other losses as raising `NotImplementError` for now.

COMMENT:
It sounds like a plan (focusing on first array API support in `sklearn/linear_model/_linear_loss.py` without touching `sklearn/_loss`)! Once this first iteration is implemented, we should profile it with py-spy or similar to check how much we have to gain from array API compatible alternatives to our Cython loss function.

Labels: High Priority, Array API
Failed installation with uv.

### Describe the bug

Trying to install scikit-learn using uv pip install scikit-learn. Python 3.14t
Got an error while building

### Steps/Code to Reproduce

Use python 3.14t
Run uv pip install scikit-learn


### Expected Results

scikit-learn successfully installed 

### Actual Results

Got an error

FAILED: [code=1] sklearn/utils/murmurhash.cp314t-win_amd64.pyd.p/meson-generated_murmurhash.c.obj
      "clang" "-Isklearn\utils\murmurhash.cp314t-win_amd64.pyd.p" "-Isklearn\utils" "-I..\sklearn\utils" "-Isklearn"
      "-IC:\Users\Egor\AppData\Local\Programs\Python\Python314\Include" "-fvisibility=hidden" "-fdiagnostics-color=always" "-DNDEBUG" "-D_FILE_OFFSET_BITS=64" "-Wall"
      "-Winvalid-pch" "-std=c11" "-O3" "-Wno-unused-but-set-variable" "-Wno-unused-function" "-Wno-conversion" "-Wno-misleading-indentation" "-DPy_GIL_DISABLED" -MD -MQ
      sklearn/utils/murmurhash.cp314t-win_amd64.pyd.p/meson-generated_murmurhash.c.obj -MF "sklearn\utils\murmurhash.cp314t-win_amd64.pyd.p\meson-generated_murmurhash.c.obj.d"    
      -o sklearn/utils/murmurhash.cp314t-win_amd64.pyd.p/meson-generated_murmurhash.c.obj "-c" sklearn/utils/murmurhash.cp314t-win_amd64.pyd.p/murmurhash.c
      In file included from sklearn/utils/murmurhash.cp314t-win_amd64.pyd.p/murmurhash.c:1129:
      ..\sklearn\utils\src/MurmurHash3.h:16:23: error: typedef redefinition with different types ('unsigned long' vs 'unsigned int')
         16 | typedef unsigned long uint32_t;
            |                       ^
      sklearn/utils/murmurhash.cp314t-win_amd64.pyd.p/murmurhash.c:468:39: note: previous definition is here
        468 |             typedef unsigned __int32  uint32_t;
            |                                       ^
      1 error generated.

### Versions

```shell
scikit-learn 1.7.2
```

COMMENT:
This happens because for scikit-learn 1.7.2 there are no Python3.14t wheels on PyPI. We have free-threaded wheels for older versions of Python, but not for the windows platform

If there is no wheel the source distribution is downloaded and then compiled. Which on windows can be quite involved. If you really want to try I think https://scikit-learn.org/dev/developers/advanced_installation.html#building-from-source is the best place to start. While that guide is for building from a Git repo, it is probably the best guide we have with regard to getting sdists to compile? Not sure I'd recommend it though.

---

@jeremiedbb do you know why there are no 3.14t wheels (for any platform)? I was assuming that if we have 3.13t and 3.14 wheels, we'd also have 3.14t?

The other question is why are there no free-threaded wheels for windows? I'm not sure, but I found https://github.com/scikit-learn/scikit-learn/issues/32393 - so maybe the answer is that it needs more work?

Labels: Bug
Failed installation with uv.

### Describe the bug

Trying to install scikit-learn using uv pip install scikit-learn. Python 3.14t
Got an error while building

### Steps/Code to Reproduce

Use python 3.14t
Run uv pip install scikit-learn


### Expected Results

scikit-learn successfully installed 

### Actual Results

Got an error

FAILED: [code=1] sklearn/utils/murmurhash.cp314t-win_amd64.pyd.p/meson-generated_murmurhash.c.obj
      "clang" "-Isklearn\utils\murmurhash.cp314t-win_amd64.pyd.p" "-Isklearn\utils" "-I..\sklearn\utils" "-Isklearn"
      "-IC:\Users\Egor\AppData\Local\Programs\Python\Python314\Include" "-fvisibility=hidden" "-fdiagnostics-color=always" "-DNDEBUG" "-D_FILE_OFFSET_BITS=64" "-Wall"
      "-Winvalid-pch" "-std=c11" "-O3" "-Wno-unused-but-set-variable" "-Wno-unused-function" "-Wno-conversion" "-Wno-misleading-indentation" "-DPy_GIL_DISABLED" -MD -MQ
      sklearn/utils/murmurhash.cp314t-win_amd64.pyd.p/meson-generated_murmurhash.c.obj -MF "sklearn\utils\murmurhash.cp314t-win_amd64.pyd.p\meson-generated_murmurhash.c.obj.d"    
      -o sklearn/utils/murmurhash.cp314t-win_amd64.pyd.p/meson-generated_murmurhash.c.obj "-c" sklearn/utils/murmurhash.cp314t-win_amd64.pyd.p/murmurhash.c
      In file included from sklearn/utils/murmurhash.cp314t-win_amd64.pyd.p/murmurhash.c:1129:
      ..\sklearn\utils\src/MurmurHash3.h:16:23: error: typedef redefinition with different types ('unsigned long' vs 'unsigned int')
         16 | typedef unsigned long uint32_t;
            |                       ^
      sklearn/utils/murmurhash.cp314t-win_amd64.pyd.p/murmurhash.c:468:39: note: previous definition is here
        468 |             typedef unsigned __int32  uint32_t;
            |                                       ^
      1 error generated.

### Versions

```shell
scikit-learn 1.7.2
```

COMMENT:
I'm not sure if this is really a bug, or more "expected behaviour". We normally provide wheels so that people don't have to build them themselves. But in this case there are no wheels :-/ So I'll leave the "bug" label for now

Labels: Bug