Talip7/scikit-learn-issues-multilabel huggingface dataset:
Describe `set_{method}_request()` API, expose `_MetadataRequester`, or expose `_BaseScorer`

### Describe the issue linked to the documentation

TL;DR: Metadata routing for scoring could either use a base class or documentation of how to write `set_score_request()`.

Currently the [Metadata Estimator Dev Guide](https://scikit-learn.org/stable/auto_examples/miscellaneous/plot_metadata_routing.html#metadata-routing) has examples of a metadata-consuming estimator and a metadata-routing estimator.  However, the metadata routing is also designed for scorers and CV splitters which may or may not be estimators.   Fortunately, `sklearn.model_selection` exposes `BaseCrossValidator`, which like `BaseEstimator`, subclasses `_MetadataRequester`.  Unfortunately, ~there's no base class for scorers.~ the base class for scorers, `_BaseScorer`, is not public.

I don't understand how to string together the relevant methods that should be a part of `set_score_params`, The current workaround is to simply subclass `BaseEstimator`, even if I'm not making an estimator, or to subclass `_MetadataRequester`, even though its not part of the public API.  ~Or use `make_scorer` to pin the kwargs when instantiating the meta-estimator, rather than in `fit()`~

My use case is for scoring a time-series model where the data generating mechanism is known to the experiment, but not the model, and I need to compare the fit model to the true data generating mechanism.  I understand how to use a custom scorer in `RandomizedSearchCV`, and the [metadata API](https://scikit-learn.org/stable/metadata_routing.html#api-interface) explains how meta estimators like `RandomizedSearchCV` can pass additional arguments to my custom scorer.

### Suggest a potential alternative/fix

* publicly exposing `_MetadataRequester`
* publicly exposing `_BaseScorer`
* Document how to write the `set_{method}_request()` methods.  It looks like `_MetadataRequester` uses a descriptor `RequestMethod`, which relies on an instance having a `_get_metadata_request` method and a `_metadata_request` attribute (which it doesn't really describe).

EDIT: Found out about `_BaseScorer`.  Also, removed `make_scorer` workaround, as it doesn't result in passing the fitted estimator to the `_score_func`

COMMENT:
@Jacob-Stevens-Haas please go ahead. Ideally you'd make everything you need either public, or a part of the _develope API_ with the `__sklearn_...__` pattern.

Labels: Documentation, Needs Investigation, Metadata Routing
Documentation improvement for macOS Homebrew libomp installation

### Describe the issue linked to the documentation

The current documentation in `doc/developers/advanced_installation.rst` under the "macOS compilers from Homebrew" section provides environment variable examples using the path `/usr/local/opt/libomp/`. While this is correct for Intel-based Macs, Homebrew on Apple Silicon (arm64) Macs installs packages, including `libomp`, to `/opt/homebrew/opt/libomp/`.

This can lead to confusion and build issues for users on Apple Silicon hardware who follow the documentation to install from source.

The documentation will improve from mentioning that `libomp` is often installed as "keg-only" by Homebrew, which is why explicitly setting these paths is necessary. Homebrew's own output (`brew info libomp`) often provides guidance on the necessary `CPPFLAGS` and `LDFLAGS`.


### Suggest a potential alternative/fix

The documentation could be updated to:
1.  Mention the different Homebrew base paths for Intel (`/usr/local`) and Apple Silicon (`/opt/homebrew`).
2.  Update the example environment variable settings to reflect the `/opt/homebrew/opt/libomp` path as a common case for Apple Silicon, or provide instructions for users to identify and use the correct path for their system.
3.  Optionally, We could briefly explain the "keg-only" nature of `libomp` from Homebrew and how it relates to needing these environment variables.

COMMENT:
/take

Labels: Documentation
Documentation improvement for macOS Homebrew libomp installation

### Describe the issue linked to the documentation

The current documentation in `doc/developers/advanced_installation.rst` under the "macOS compilers from Homebrew" section provides environment variable examples using the path `/usr/local/opt/libomp/`. While this is correct for Intel-based Macs, Homebrew on Apple Silicon (arm64) Macs installs packages, including `libomp`, to `/opt/homebrew/opt/libomp/`.

This can lead to confusion and build issues for users on Apple Silicon hardware who follow the documentation to install from source.

The documentation will improve from mentioning that `libomp` is often installed as "keg-only" by Homebrew, which is why explicitly setting these paths is necessary. Homebrew's own output (`brew info libomp`) often provides guidance on the necessary `CPPFLAGS` and `LDFLAGS`.


### Suggest a potential alternative/fix

The documentation could be updated to:
1.  Mention the different Homebrew base paths for Intel (`/usr/local`) and Apple Silicon (`/opt/homebrew`).
2.  Update the example environment variable settings to reflect the `/opt/homebrew/opt/libomp` path as a common case for Apple Silicon, or provide instructions for users to identify and use the correct path for their system.
3.  Optionally, We could briefly explain the "keg-only" nature of `libomp` from Homebrew and how it relates to needing these environment variables.

COMMENT:
I guess this section can likely be simplified indeed.

As mentioned in https://github.com/scikit-learn/scikit-learn/pull/31361#issuecomment-2876202394 I there is likely no need to set environment variables now that we use Meson but this needs to be double-checked.

Labels: Documentation
Documentation improvement for macOS Homebrew libomp installation

### Describe the issue linked to the documentation

The current documentation in `doc/developers/advanced_installation.rst` under the "macOS compilers from Homebrew" section provides environment variable examples using the path `/usr/local/opt/libomp/`. While this is correct for Intel-based Macs, Homebrew on Apple Silicon (arm64) Macs installs packages, including `libomp`, to `/opt/homebrew/opt/libomp/`.

This can lead to confusion and build issues for users on Apple Silicon hardware who follow the documentation to install from source.

The documentation will improve from mentioning that `libomp` is often installed as "keg-only" by Homebrew, which is why explicitly setting these paths is necessary. Homebrew's own output (`brew info libomp`) often provides guidance on the necessary `CPPFLAGS` and `LDFLAGS`.


### Suggest a potential alternative/fix

The documentation could be updated to:
1.  Mention the different Homebrew base paths for Intel (`/usr/local`) and Apple Silicon (`/opt/homebrew`).
2.  Update the example environment variable settings to reflect the `/opt/homebrew/opt/libomp` path as a common case for Apple Silicon, or provide instructions for users to identify and use the correct path for their system.
3.  Optionally, We could briefly explain the "keg-only" nature of `libomp` from Homebrew and how it relates to needing these environment variables.

COMMENT:
Actually I did not remember but I created an issue saying that we can probably remove the environment variables a while ago https://github.com/scikit-learn/scikit-learn/issues/29603.

Labels: Documentation
Documentation improvement for macOS Homebrew libomp installation

### Describe the issue linked to the documentation

The current documentation in `doc/developers/advanced_installation.rst` under the "macOS compilers from Homebrew" section provides environment variable examples using the path `/usr/local/opt/libomp/`. While this is correct for Intel-based Macs, Homebrew on Apple Silicon (arm64) Macs installs packages, including `libomp`, to `/opt/homebrew/opt/libomp/`.

This can lead to confusion and build issues for users on Apple Silicon hardware who follow the documentation to install from source.

The documentation will improve from mentioning that `libomp` is often installed as "keg-only" by Homebrew, which is why explicitly setting these paths is necessary. Homebrew's own output (`brew info libomp`) often provides guidance on the necessary `CPPFLAGS` and `LDFLAGS`.


### Suggest a potential alternative/fix

The documentation could be updated to:
1.  Mention the different Homebrew base paths for Intel (`/usr/local`) and Apple Silicon (`/opt/homebrew`).
2.  Update the example environment variable settings to reflect the `/opt/homebrew/opt/libomp` path as a common case for Apple Silicon, or provide instructions for users to identify and use the correct path for their system.
3.  Optionally, We could briefly explain the "keg-only" nature of `libomp` from Homebrew and how it relates to needing these environment variables.

COMMENT:
Okay, cool. Thanks for confirming. I will update the PR 👍

Labels: Documentation
Benchmark Function

### Describe the workflow you want to enable

I would like to define multiple pipelines and compare them against each other on multiple datasets.

### Describe your proposed solution

A single helper function that executes this benchmark fully in parallel. This would allow 

### Describe alternatives you've considered, if relevant

There is an [MLR3 function](https://mlr3.mlr-org.com/reference/benchmark.html) that inspired this issue. 

### Additional context

Reasoning: I'm currently co-teaching a course where students can do the exercises in R using MLR3 or Python using scikit-learn. Doing the exercises in R appears to be less repetitive overall, as for example, there is a simple function for benchmarking. Also, it would require less time to actually wait for the results to finish as one could make more use of parallelism.

COMMENT:
You can already simply do that by creating your `Pipeline` objects, and passing them to a `cross_val_score` in a look over your datasets and pipelines. Since the code snippet required to do this is just a few lines, I don't think it makes sense for us to add it to the public API.

Labels: New Feature, Needs Triage
Add Multiple Kernel Learning (MKL) for Support Vector Machines (SVM)

### Describe the workflow you want to enable

I propose adding a [Multiple Kernel Learning (MKL)](https://en.wikipedia.org/wiki/Multiple_kernel_learning) module for kernel optimization in kernel-based methods (such as SVM) to scikit-learn. MKL is a more advanced approach compared to GridSearchCV, offering a way to combine multiple kernels into a single, optimal kernel. In the worst case, MKL will behave like GridSearchCV by assigning a weight of 1 to the best kernel, but in the other cases, it will provide a weighted combination of kernels for better generalization.

### Describe your proposed solution

I have already implemented a complete MKL solution for regression, binary and multi-class classification, and clustering (One-Class). This implementation includes the [SimpleMKL algorithm](https://www.jmlr.org/papers/volume9/rakotomamonjy08a/rakotomamonjy08a.pdf), which optimizes the weights of the kernels, as well as the AverageMKL (simply averages the kernels) and SumMKL (simply sums the kernels) algorithms. This implementation is available on a [previously closed pull request](https://github.com/scikit-learn/scikit-learn/pull/31166).

### Describe alternatives you've considered, if relevant

An alternative would be to continue relying on GridSearchCV for kernel selection. However, GridSearchCV is limited to selecting only one kernel and does not consider the possibility of combining multiple kernels, which can result in suboptimal performance. MKL provides a more sophisticated approach by optimizing kernel weights, leading to better performance in many machine learning tasks.

COMMENT:
/take

Labels: New Feature, Needs Decision
Add Multiple Kernel Learning (MKL) for Support Vector Machines (SVM)

### Describe the workflow you want to enable

I propose adding a [Multiple Kernel Learning (MKL)](https://en.wikipedia.org/wiki/Multiple_kernel_learning) module for kernel optimization in kernel-based methods (such as SVM) to scikit-learn. MKL is a more advanced approach compared to GridSearchCV, offering a way to combine multiple kernels into a single, optimal kernel. In the worst case, MKL will behave like GridSearchCV by assigning a weight of 1 to the best kernel, but in the other cases, it will provide a weighted combination of kernels for better generalization.

### Describe your proposed solution

I have already implemented a complete MKL solution for regression, binary and multi-class classification, and clustering (One-Class). This implementation includes the [SimpleMKL algorithm](https://www.jmlr.org/papers/volume9/rakotomamonjy08a/rakotomamonjy08a.pdf), which optimizes the weights of the kernels, as well as the AverageMKL (simply averages the kernels) and SumMKL (simply sums the kernels) algorithms. This implementation is available on a [previously closed pull request](https://github.com/scikit-learn/scikit-learn/pull/31166).

### Describe alternatives you've considered, if relevant

An alternative would be to continue relying on GridSearchCV for kernel selection. However, GridSearchCV is limited to selecting only one kernel and does not consider the possibility of combining multiple kernels, which can result in suboptimal performance. MKL provides a more sophisticated approach by optimizing kernel weights, leading to better performance in many machine learning tasks.

COMMENT:
@Nikita2812 this issue is to discuss inclusion of a feature, and the author already has a PR implementing it. So I'm not sure what it is that you're "take"ing. Please read the issues carefully and assess the situation before claiming them.

@thomasbauer76 a few of us had a mini-discussion about this last week. The citation counts certainly pass our inclusion criteria, and it certainly improves upon standard single kernel SVMs. However, like SVMs, it's usually quite hard to find a nice kernel for the data at hand. In the modern age, we're not sure about including this in the library, due to the balance between potential usage, and maintenance cost.

On the plus side, this effort can update our existing vendored libsvm, which can be a blessing. But that can be done w/o having to include MKL (or updating it might make including MKL almost trivial), and we welcome any efforts trying to update our libsvm. It could even potentially be a git submodule which we'd update as the upstream updates.

cc @scikit-learn/core-devs for visibility and more opinions here.

Labels: New Feature, Needs Decision
Add Multiple Kernel Learning (MKL) for Support Vector Machines (SVM)

### Describe the workflow you want to enable

I propose adding a [Multiple Kernel Learning (MKL)](https://en.wikipedia.org/wiki/Multiple_kernel_learning) module for kernel optimization in kernel-based methods (such as SVM) to scikit-learn. MKL is a more advanced approach compared to GridSearchCV, offering a way to combine multiple kernels into a single, optimal kernel. In the worst case, MKL will behave like GridSearchCV by assigning a weight of 1 to the best kernel, but in the other cases, it will provide a weighted combination of kernels for better generalization.

### Describe your proposed solution

I have already implemented a complete MKL solution for regression, binary and multi-class classification, and clustering (One-Class). This implementation includes the [SimpleMKL algorithm](https://www.jmlr.org/papers/volume9/rakotomamonjy08a/rakotomamonjy08a.pdf), which optimizes the weights of the kernels, as well as the AverageMKL (simply averages the kernels) and SumMKL (simply sums the kernels) algorithms. This implementation is available on a [previously closed pull request](https://github.com/scikit-learn/scikit-learn/pull/31166).

### Describe alternatives you've considered, if relevant

An alternative would be to continue relying on GridSearchCV for kernel selection. However, GridSearchCV is limited to selecting only one kernel and does not consider the possibility of combining multiple kernels, which can result in suboptimal performance. MKL provides a more sophisticated approach by optimizing kernel weights, leading to better performance in many machine learning tasks.

COMMENT:
> [@Nikita2812](https://github.com/Nikita2812) this issue is to discuss inclusion of a feature, and the author already has a PR implementing it. So I'm not sure what it is that you're "take"ing. Please read the issues carefully and assess the situation before claiming them.
> 
> [@thomasbauer76](https://github.com/thomasbauer76) a few of us had a mini-discussion about this last week. The citation counts certainly pass our inclusion criteria, and it certainly improves upon standard single kernel SVMs. However, like SVMs, it's usually quite hard to find a nice kernel for the data at hand. In the modern age, we're not sure about including this in the library, due to the balance between potential usage, and maintenance cost.
> 
> On the plus side, this effort can update our existing vendored libsvm, which can be a blessing. But that can be done w/o having to include MKL (or updating it might make including MKL almost trivial), and we welcome any efforts trying to update our libsvm. It could even potentially be a git submodule which we'd update as the upstream updates.
> 
> cc @scikit-learn/core-devs for visibility and more opinions here.

Sorry for the late answer.

Thanks for the detailed feedback. I totally understand the concerns regarding maintenance and usage.

If I find some free time, I’d be happy to look into updating the vendored libsvm by adding alpha seeding!

Labels: New Feature, Needs Decision
Add Multiple Kernel Learning (MKL) for Support Vector Machines (SVM)

### Describe the workflow you want to enable

I propose adding a [Multiple Kernel Learning (MKL)](https://en.wikipedia.org/wiki/Multiple_kernel_learning) module for kernel optimization in kernel-based methods (such as SVM) to scikit-learn. MKL is a more advanced approach compared to GridSearchCV, offering a way to combine multiple kernels into a single, optimal kernel. In the worst case, MKL will behave like GridSearchCV by assigning a weight of 1 to the best kernel, but in the other cases, it will provide a weighted combination of kernels for better generalization.

### Describe your proposed solution

I have already implemented a complete MKL solution for regression, binary and multi-class classification, and clustering (One-Class). This implementation includes the [SimpleMKL algorithm](https://www.jmlr.org/papers/volume9/rakotomamonjy08a/rakotomamonjy08a.pdf), which optimizes the weights of the kernels, as well as the AverageMKL (simply averages the kernels) and SumMKL (simply sums the kernels) algorithms. This implementation is available on a [previously closed pull request](https://github.com/scikit-learn/scikit-learn/pull/31166).

### Describe alternatives you've considered, if relevant

An alternative would be to continue relying on GridSearchCV for kernel selection. However, GridSearchCV is limited to selecting only one kernel and does not consider the possibility of combining multiple kernels, which can result in suboptimal performance. MKL provides a more sophisticated approach by optimizing kernel weights, leading to better performance in many machine learning tasks.

COMMENT:
> If I find some free time, I’d be happy to look into updating the vendored libsvm by adding alpha seeding!

What a helpful and positive answer. Thank you so much!

Labels: New Feature, Needs Decision
⚠️ CI failed on Wheel builder (last failure: May 10, 2025) ⚠️

**CI failed on [Wheel builder](https://github.com/scikit-learn/scikit-learn/actions/runs/14941597365)** (May 10, 2025)

COMMENT:
## CI is no longer failing! ✅

[Successful run](https://github.com/scikit-learn/scikit-learn/actions/runs/14952022634) on May 11, 2025

Labels: Needs Triage
Add MultiHorizonTimeSeriesSplit for Multi-Horizon Time Series Cross-Validation

### Describe the workflow you want to enable

The current `TimeSeriesSplit` in scikit-learn supports cross-validation for time series data with a single prediction horizon per split, which limits its use for scenarios requiring forecasts over multiple future steps (e.g., predicting 1, 3, and 5 days ahead). I propose adding a new class, `MultiHorizonTimeSeriesSplit`, to enable cross-validation with multiple prediction horizons in a single split.

This would allow users to:
- Specify a list of horizons (e.g., `[1, 3, 5]`) to generate train-test splits where the test set includes indices for multiple future steps.
- Evaluate time series models for short, medium, and long-term forecasts simultaneously.
- Simplify workflows for applications like demand forecasting, financial modeling, or weather prediction, avoiding manual splitting.

Example usage with daily temperatures:
```
from sklearn.model_selection import MultiHorizonTimeSeriesSplit
import numpy as np

# Daily temperatures for 10 days (in °C)
X = np.array([20, 21, 22, 23, 24, 25, 26, 27, 28, 29])
cv = MultiHorizonTimeSeriesSplit(n_splits=2, horizons=[1, 2])
for train_idx, test_idx in cv.split(X):
    print(f"Train indices: {train_idx}, Test indices: {test_idx}")
```
Expected output:
```
Train indices: [0 1 2 3 4], Test indices: [5 6]
Train indices: [0 1 2 3 4 5 6], Test indices: [7 8]
```

### Describe your proposed solution

I propose implementing a new class, `MultiHorizonTimeSeriesSplit`, inheriting from `TimeSeriesSplit`. The class will:
- Add a `horizons` parameter (list of integers) to specify prediction steps.
- Modify the `split` method to generate test indices for each horizon while preserving temporal order.
- Include input validation to ensure valid horizons and splits.

To ensure the correctness of MultiHorizonTimeSeriesSplit, we will develop unit tests covering various configurations and edge cases. For benchmarking, we will assess the computational efficiency and correctness of the new class compared to manual splitting. We will use synthetic time series to evaluate scalability and measure split generation time and memory usage, running tests on a personal laptop.


### Describe alternatives you've considered, if relevant

_No response_

### Additional context

_No response_

COMMENT:
/take

Labels: New Feature