Talip7/scikit-learn-issues-multilabel huggingface dataset:
RFC Suggesting HistGradientBoosting in RandomForest and GradientBoosting pages

Right now we have this in the GradientBoosting API page:

> [sklearn.ensemble.HistGradientBoostingClassifier](https://scikit-learn.org/dev/modules/generated/sklearn.ensemble.HistGradientBoostingClassifier.html#sklearn.ensemble.HistGradientBoostingClassifier) is a much faster variant of this algorithm for intermediate datasets (n_samples >= 10_000).

After LogisticRegression and LinearRegression, RandomForest is the third most visited page among models we have. And I think we'd agree that in most cases users can probably be better off using HGBT models instead. Right now users compare random forests with xgboost, while they could be using HGBT.

So my question is, should we add a message on forest/tree pages regarding HGBT, and have the statement a bit bolder than just saying "it's probably faster"?

@amueller had done quite a bit of analysis on this, maybe we could link to those?

COMMENT:
@adrinjalali You are absolutely right. And that is the reason why I hesitate. You are arguing more from an advertising perspective which in my opinion should not be placed in the docstring. (As a reductio ad absurdum, it would be awkward to mention it in the docstring of all of our linear models.)

Let's us better advertise our HGBT models: in the user guide, in blog posts, in examples etc. but not in the docstrings.

Labels: Documentation, RFC
RFC Suggesting HistGradientBoosting in RandomForest and GradientBoosting pages

Right now we have this in the GradientBoosting API page:

> [sklearn.ensemble.HistGradientBoostingClassifier](https://scikit-learn.org/dev/modules/generated/sklearn.ensemble.HistGradientBoostingClassifier.html#sklearn.ensemble.HistGradientBoostingClassifier) is a much faster variant of this algorithm for intermediate datasets (n_samples >= 10_000).

After LogisticRegression and LinearRegression, RandomForest is the third most visited page among models we have. And I think we'd agree that in most cases users can probably be better off using HGBT models instead. Right now users compare random forests with xgboost, while they could be using HGBT.

So my question is, should we add a message on forest/tree pages regarding HGBT, and have the statement a bit bolder than just saying "it's probably faster"?

@amueller had done quite a bit of analysis on this, maybe we could link to those?

COMMENT:
I agree all your suggestions are good things, but it won't have much impact if the users are not seeing them. We know where people look, and we should make them aware by putting the information where people go, not just where we think they should go.

Labels: Documentation, RFC
RFC Suggesting HistGradientBoosting in RandomForest and GradientBoosting pages

Right now we have this in the GradientBoosting API page:

> [sklearn.ensemble.HistGradientBoostingClassifier](https://scikit-learn.org/dev/modules/generated/sklearn.ensemble.HistGradientBoostingClassifier.html#sklearn.ensemble.HistGradientBoostingClassifier) is a much faster variant of this algorithm for intermediate datasets (n_samples >= 10_000).

After LogisticRegression and LinearRegression, RandomForest is the third most visited page among models we have. And I think we'd agree that in most cases users can probably be better off using HGBT models instead. Right now users compare random forests with xgboost, while they could be using HGBT.

So my question is, should we add a message on forest/tree pages regarding HGBT, and have the statement a bit bolder than just saying "it's probably faster"?

@amueller had done quite a bit of analysis on this, maybe we could link to those?

COMMENT:
Then just add it to „see also“ and maybe link to @ArturoAmorQ‘s new example.

Labels: Documentation, RFC
RFC Suggesting HistGradientBoosting in RandomForest and GradientBoosting pages

Right now we have this in the GradientBoosting API page:

> [sklearn.ensemble.HistGradientBoostingClassifier](https://scikit-learn.org/dev/modules/generated/sklearn.ensemble.HistGradientBoostingClassifier.html#sklearn.ensemble.HistGradientBoostingClassifier) is a much faster variant of this algorithm for intermediate datasets (n_samples >= 10_000).

After LogisticRegression and LinearRegression, RandomForest is the third most visited page among models we have. And I think we'd agree that in most cases users can probably be better off using HGBT models instead. Right now users compare random forests with xgboost, while they could be using HGBT.

So my question is, should we add a message on forest/tree pages regarding HGBT, and have the statement a bit bolder than just saying "it's probably faster"?

@amueller had done quite a bit of analysis on this, maybe we could link to those?

COMMENT:
I had a look at https://blog.scikit-learn.org/ and couldn't find a post about HGBT. Might be a good idea to take one of our shorter examples about HGBT (vs RF or just about HGBT) and posting it as a blog post. Every little helps

Labels: Documentation, RFC
RFC Suggesting HistGradientBoosting in RandomForest and GradientBoosting pages

Right now we have this in the GradientBoosting API page:

> [sklearn.ensemble.HistGradientBoostingClassifier](https://scikit-learn.org/dev/modules/generated/sklearn.ensemble.HistGradientBoostingClassifier.html#sklearn.ensemble.HistGradientBoostingClassifier) is a much faster variant of this algorithm for intermediate datasets (n_samples >= 10_000).

After LogisticRegression and LinearRegression, RandomForest is the third most visited page among models we have. And I think we'd agree that in most cases users can probably be better off using HGBT models instead. Right now users compare random forests with xgboost, while they could be using HGBT.

So my question is, should we add a message on forest/tree pages regarding HGBT, and have the statement a bit bolder than just saying "it's probably faster"?

@amueller had done quite a bit of analysis on this, maybe we could link to those?

COMMENT:
I think all the points in this issue have been addressed, or am I missing something?

Labels: Documentation, RFC
RFC Suggesting HistGradientBoosting in RandomForest and GradientBoosting pages

Right now we have this in the GradientBoosting API page:

> [sklearn.ensemble.HistGradientBoostingClassifier](https://scikit-learn.org/dev/modules/generated/sklearn.ensemble.HistGradientBoostingClassifier.html#sklearn.ensemble.HistGradientBoostingClassifier) is a much faster variant of this algorithm for intermediate datasets (n_samples >= 10_000).

After LogisticRegression and LinearRegression, RandomForest is the third most visited page among models we have. And I think we'd agree that in most cases users can probably be better off using HGBT models instead. Right now users compare random forests with xgboost, while they could be using HGBT.

So my question is, should we add a message on forest/tree pages regarding HGBT, and have the statement a bit bolder than just saying "it's probably faster"?

@amueller had done quite a bit of analysis on this, maybe we could link to those?

COMMENT:
I really think our statements are not strong enough in our documentation, and it looks like we're saying there's also this other thing that people _could_ use, but what we really wanna say is people _should_ use HGBT instead. I'm happy closing this issue, but I don't think we've achieved the goal.

Labels: Documentation, RFC
RFC Suggesting HistGradientBoosting in RandomForest and GradientBoosting pages

Right now we have this in the GradientBoosting API page:

> [sklearn.ensemble.HistGradientBoostingClassifier](https://scikit-learn.org/dev/modules/generated/sklearn.ensemble.HistGradientBoostingClassifier.html#sklearn.ensemble.HistGradientBoostingClassifier) is a much faster variant of this algorithm for intermediate datasets (n_samples >= 10_000).

After LogisticRegression and LinearRegression, RandomForest is the third most visited page among models we have. And I think we'd agree that in most cases users can probably be better off using HGBT models instead. Right now users compare random forests with xgboost, while they could be using HGBT.

So my question is, should we add a message on forest/tree pages regarding HGBT, and have the statement a bit bolder than just saying "it's probably faster"?

@amueller had done quite a bit of analysis on this, maybe we could link to those?

COMMENT:
> I really think our statements are not strong enough in our documentation, and it looks like we're saying there's also this other thing that people _could_ use, but what we really wanna say is people _should_ use HGBT instead.

Let's try to take the occasion presented in #26826 to do a better job in sending such message.

Labels: Documentation, RFC
Possible inconsistency between documentation and code for user guide of PLSCanonical

### Describe the issue linked to the documentation

Recently I've been learning Partial Least Squares (PLS), and learned that there are some variations of PLS, namely PLS-Canonical, PLS-SVD, PLS2 and PLS1 (mainly from the [User Guide of cross decomposition](https://scikit-learn.org/stable/modules/cross_decomposition.html#cross-decompositionhttps://) in the python's scikit-learn library. I could understand most of it, and have read the reference that page has provided [A survey of Partial Least Squares (PLS) methods, with emphasis on the two-block case, by JA Wegelin](https://stat.uw.edu/sites/default/files/files/reports/2000/tr371.pdf). Nevertheless, I still have questions about the details of predicting $Y$ based on new samples.

In the user guide, it states that to compute $Y$, we need 

> The idea is to try to predict the transformed targets $\Omega$ as a function of the transformed samples $\Xi$, by computing $\alpha \in \mathbb{R}$ such that $\Omega = \alpha \Xi$.
>
> ...
>
> ... **and as a result the coefficient matrix $\beta = \alpha P \Delta^T$**.

However in code, `self._coef_` is computed by
```python
self._coef_ = np.dot(self.x_rotations_, self.y_loadings_.T)
self._coef_ = (self._coef_ * self._y_std).T
```
This code is located at [L355](https://github.com/scikit-learn/scikit-learn/blob/main/sklearn/cross_decomposition/_pls.py#L355) of `_pls.py`.

If I am correct, here `self.x_rotations_` is a transformation matrix such that `X @ self.x_rotations_` would be $\Xi$. Therefore `self.x_rotations_` would be $P$. And `self.y_loadings_` is $\Delta$. **As a result, the code is computing $\beta = P \Delta^T $, which I think is inconsistent with the documentation described above.**


### Suggest a potential alternative/fix

In my honest opinion, maybe we should add the computation of $\alpha$ into the code. 

Moreover, I think $\alpha$ shouldn't be a scalar; instead, $\alpha$ should be a vector of length $R$, where $R$ is the number of columns of $\Xi$, i.e. the number of latent scores. Each $\xi_i$ should be multiplied by $\alpha_i$ before multiplying $\Delta^T$:

$$ 
\beta = P 
\left[ \begin{matrix}
 \alpha_1 & & \\
& \ddots & \\
& & \alpha_R 
 \end{matrix}  \right]
 \Delta^T 
$$

COMMENT:
Thanks for the report. This needs more investigation, cc @scikit-learn/core-devs in case somebody has it active in their mind and could respond.

Labels: Documentation, Needs Investigation
Possible inconsistency between documentation and code for user guide of PLSCanonical

### Describe the issue linked to the documentation

Recently I've been learning Partial Least Squares (PLS), and learned that there are some variations of PLS, namely PLS-Canonical, PLS-SVD, PLS2 and PLS1 (mainly from the [User Guide of cross decomposition](https://scikit-learn.org/stable/modules/cross_decomposition.html#cross-decompositionhttps://) in the python's scikit-learn library. I could understand most of it, and have read the reference that page has provided [A survey of Partial Least Squares (PLS) methods, with emphasis on the two-block case, by JA Wegelin](https://stat.uw.edu/sites/default/files/files/reports/2000/tr371.pdf). Nevertheless, I still have questions about the details of predicting $Y$ based on new samples.

In the user guide, it states that to compute $Y$, we need 

> The idea is to try to predict the transformed targets $\Omega$ as a function of the transformed samples $\Xi$, by computing $\alpha \in \mathbb{R}$ such that $\Omega = \alpha \Xi$.
>
> ...
>
> ... **and as a result the coefficient matrix $\beta = \alpha P \Delta^T$**.

However in code, `self._coef_` is computed by
```python
self._coef_ = np.dot(self.x_rotations_, self.y_loadings_.T)
self._coef_ = (self._coef_ * self._y_std).T
```
This code is located at [L355](https://github.com/scikit-learn/scikit-learn/blob/main/sklearn/cross_decomposition/_pls.py#L355) of `_pls.py`.

If I am correct, here `self.x_rotations_` is a transformation matrix such that `X @ self.x_rotations_` would be $\Xi$. Therefore `self.x_rotations_` would be $P$. And `self.y_loadings_` is $\Delta$. **As a result, the code is computing $\beta = P \Delta^T $, which I think is inconsistent with the documentation described above.**


### Suggest a potential alternative/fix

In my honest opinion, maybe we should add the computation of $\alpha$ into the code. 

Moreover, I think $\alpha$ shouldn't be a scalar; instead, $\alpha$ should be a vector of length $R$, where $R$ is the number of columns of $\Xi$, i.e. the number of latent scores. Each $\xi_i$ should be multiplied by $\alpha_i$ before multiplying $\Delta^T$:

$$ 
\beta = P 
\left[ \begin{matrix}
 \alpha_1 & & \\
& \ddots & \\
& & \alpha_R 
 \end{matrix}  \right]
 \Delta^T 
$$

COMMENT:
ping @duchesnay

Labels: Documentation, Needs Investigation
confusing fit failure when using set_config(transform_output="pandas")

### Describe the bug

Using `set_config(transform_output="pandas")` in combination with `ColumnTransformer` may result in a failed fit, since indices do not align as expected. The error will not be especially helpful: "ValueError: Found input variables with inconsistent numbers of samples: [705, 527]" (Error appears to be related to merging DataFrames with non-aligned indices, as evident by NaNs and change in shape).

### Steps/Code to Reproduce

```
import pandas as pd
from sklearn.ensemble import GradientBoostingRegressor
from sklearn.pipeline import Pipeline
from sklearn.decomposition import TruncatedSVD
from sklearn import set_config
from sklearn.feature_extraction.text import CountVectorizer

set_config(transform_output="pandas")

d = pd.DataFrame({
    'txt': ['cat eats milk', 'tree says hi', 'bone eats dog'],
}, index=[8, 5, 3])
d


Pipeline([('cv', CountVectorizer()), ('tsvd', TruncatedSVD(n_components=1))]).fit_transform(d.txt)


```

### Expected Results

expected index: [8, 5, 3]

expected behavior: works with ColumnTransformer

### Actual Results

actual index: [0, 1, 2]

actual behavior: doesn't work with ColumnTransformer if other transformers maintain aligned indices

### Versions

```shell
System:
    python: 3.8.16 | packaged by conda-forge | (default, Feb  1 2023, 16:01:55)  [GCC 11.3.0]
executable: /opt/conda/envs/py38/bin/python3.8
   machine: Linux-4.19.0-23-cloud-amd64-x86_64-with-glibc2.10

Python dependencies:
      sklearn: 1.2.2
          pip: 23.0.1
   setuptools: 67.6.1
        numpy: 1.23.5
        scipy: 1.10.1
       Cython: None
       pandas: 1.5.3
   matplotlib: 3.7.1
       joblib: 1.2.0
threadpoolctl: 3.1.0

Built with OpenMP: True

threadpoolctl info:
       user_api: blas
   internal_api: openblas
         prefix: libopenblas
       filepath: /opt/conda/envs/py38/lib/python3.8/site-packages/numpy.libs/libopenblas64_p-r0-742d56dc.3.20.so
        version: 0.3.20
threading_layer: pthreads
   architecture: Haswell
    num_threads: 4

       user_api: openmp
   internal_api: openmp
         prefix: libgomp
       filepath: /opt/conda/envs/py38/lib/python3.8/site-packages/scikit_learn.libs/libgomp-a34b3233.so.1.0.0
        version: None
    num_threads: 4

       user_api: blas
   internal_api: openblas
         prefix: libopenblas
       filepath: /opt/conda/envs/py38/lib/python3.8/site-packages/scipy.libs/libopenblasp-r0-41284840.3.18.so
        version: 0.3.18
threading_layer: pthreads
   architecture: Haswell
    num_threads: 4
```

COMMENT:
I'd like to try my hand here, even though I'm not sure if I can solve that. By Friday I'll be able to tell. 
Is this okay @ogrisel?

Labels: Bug, module:compose, Pandas compatibility
confusing fit failure when using set_config(transform_output="pandas")

### Describe the bug

Using `set_config(transform_output="pandas")` in combination with `ColumnTransformer` may result in a failed fit, since indices do not align as expected. The error will not be especially helpful: "ValueError: Found input variables with inconsistent numbers of samples: [705, 527]" (Error appears to be related to merging DataFrames with non-aligned indices, as evident by NaNs and change in shape).

### Steps/Code to Reproduce

```
import pandas as pd
from sklearn.ensemble import GradientBoostingRegressor
from sklearn.pipeline import Pipeline
from sklearn.decomposition import TruncatedSVD
from sklearn import set_config
from sklearn.feature_extraction.text import CountVectorizer

set_config(transform_output="pandas")

d = pd.DataFrame({
    'txt': ['cat eats milk', 'tree says hi', 'bone eats dog'],
}, index=[8, 5, 3])
d


Pipeline([('cv', CountVectorizer()), ('tsvd', TruncatedSVD(n_components=1))]).fit_transform(d.txt)


```

### Expected Results

expected index: [8, 5, 3]

expected behavior: works with ColumnTransformer

### Actual Results

actual index: [0, 1, 2]

actual behavior: doesn't work with ColumnTransformer if other transformers maintain aligned indices

### Versions

```shell
System:
    python: 3.8.16 | packaged by conda-forge | (default, Feb  1 2023, 16:01:55)  [GCC 11.3.0]
executable: /opt/conda/envs/py38/bin/python3.8
   machine: Linux-4.19.0-23-cloud-amd64-x86_64-with-glibc2.10

Python dependencies:
      sklearn: 1.2.2
          pip: 23.0.1
   setuptools: 67.6.1
        numpy: 1.23.5
        scipy: 1.10.1
       Cython: None
       pandas: 1.5.3
   matplotlib: 3.7.1
       joblib: 1.2.0
threadpoolctl: 3.1.0

Built with OpenMP: True

threadpoolctl info:
       user_api: blas
   internal_api: openblas
         prefix: libopenblas
       filepath: /opt/conda/envs/py38/lib/python3.8/site-packages/numpy.libs/libopenblas64_p-r0-742d56dc.3.20.so
        version: 0.3.20
threading_layer: pthreads
   architecture: Haswell
    num_threads: 4

       user_api: openmp
   internal_api: openmp
         prefix: libgomp
       filepath: /opt/conda/envs/py38/lib/python3.8/site-packages/scikit_learn.libs/libgomp-a34b3233.so.1.0.0
        version: None
    num_threads: 4

       user_api: blas
   internal_api: openblas
         prefix: libopenblas
       filepath: /opt/conda/envs/py38/lib/python3.8/site-packages/scipy.libs/libopenblasp-r0-41284840.3.18.so
        version: 0.3.18
threading_layer: pthreads
   architecture: Haswell
    num_threads: 4
```

COMMENT:
Thanks for giving me the opportunity. Unfortunately I wasn't able to put enough time aside to investigate this and now I'll be busy with other deadlined tasks for the next 5 days at least. Thus, I cannot post a 'take'.

Labels: Bug, module:compose, Pandas compatibility
CircleCI token in README broken

### Describe the issue linked to the documentation

<img src="https://user-images.githubusercontent.com/108576690/232124886-02b5c01c-2077-44ea-a7b6-6a23ec355e4f.png" width="80%">

As is shown above, it seems that the CircleCI token is not generating correctly.


### Suggest a potential alternative/fix

_No response_

COMMENT:
Thanks for the report. This should be fixed now.

Labels: Documentation, Needs Triage
SLEP006: default routing

In the context of: 
- https://github.com/scikit-learn/scikit-learn/issues/25776
- https://github.com/scikit-learn/scikit-learn/issues/26050
 
 we've also discussed the possibility of developing a default routing strategy for certain metadata. In most cases this is `sample_weight` and probably `groups` in scikit-learn itself.

This would mean, after the introduction of SLEP006, this code would work, and route `sample_weight` to every object which accepts it (since that's what we think is suitable in this case):

```py
# The exact API is TBD
sklearn.set_config(enable_auto_routing=True)

GridSearchCV(
	LogisticRegression(), 
	scorer=a_scorer_supporting_sample_weight, ...
).fit(..., sample_weight=sw)
```

We can then decide whether or not we want the auto-routing to be enabled by default. One major thing to consider here is that with auto-routing enabled, behavior of the same code can change from version to version for two main reasons:

- we change our mind / find bugs / etc in the routing, and how we want to route things
- estimator A might not support `sample_weight` in version `x`, but starts supporting it in version `x+1`, and with default routing the behavior of the same code changes

## Notes
- auto-routing can always be overridden by the user, for more advanced usecases.
- third party developers can use the same mechanism, for `sample_weight` or other metadata if they see fit
- we might deem https://github.com/scikit-learn/scikit-learn/issues/26050 unnecessary if we develop this feature

cc @scikit-learn/core-devs

COMMENT:
For the case of `sample_weight` I think:

- the general rule would be to ensure that the resulting estimator implements the [reweightnig/repetition equivalence described in the glossary](https://scikit-learn.org/dev/glossary.html#term-sample_weight) (see also: #30564),
- it should be always routed to sub-estimators by `*SearchCV`, `Pipeline`, `FeatureUnion` and `ColumnTransformer`,
- if a pipeline is fitted with weights, but some step does not accept `sample_weight`, we should find a way to warn the user that the result might not be correct from a statistical point of view,
- most estimators with a `scoring` parameter (e.g. `*SearchCV`) should route the weights to the underlying scorer.

Some estimators such as `BaggingClassifier/Regressor` should consume the weights to do the resampling but not broadcast to the underlying estimator. We could allow the user to override this (e.g. perform uniform resampling + route the weights to the underlying estimator) but I am not sure how to express that with the routing API only. 

There exist meta-estimators such as `RANSACRegressor` that are even trickier: at a different points its `fit` method, it either iteratively fits the sub-estimator without weight but under a weight-aware resampled training set or fits the sub-estimator with weights on the final outlier filtered but otherwise non-resampled dataset (#15836). Furthermore, the meta-estimator fit needs to compare weight-aware score values, but those score functions should not be passed the weights when computed on a weight-aware resampled dataset.

Labels: API
⚠️ CI failed on Ubuntu_Atlas.ubuntu_atlas ⚠️

**CI failed on [Ubuntu_Atlas.ubuntu_atlas](https://dev.azure.com/scikit-learn/scikit-learn/_build/results?buildId=54164&view=logs&j=689a1c8f-ff4e-5689-1a1a-6fa551ae9eba)** (Apr 14, 2023)
- test_float_precision[33-MiniBatchKMeans-dense]

COMMENT:
## CI is no longer failing! ✅

[Successful run](https://dev.azure.com/scikit-learn/scikit-learn/_build/results?buildId=54226&view=logs&j=689a1c8f-ff4e-5689-1a1a-6fa551ae9eba) on Apr 15, 2023

Labels: Needs Triage