Talip7/scikit-learn-issues-multilabel huggingface dataset:
_safe_indexing being the only *private* thing listed in classes.rst

`_safe_indexing` is the only _private_ tool we have listed in `classes.rst`.

`_safe_indexing` was also mentioned in https://github.com/scikit-learn/scikit-learn/issues/15801, but this issue is to have the discussion about this very limited scope.

We should really either remove it from the docs, or make it public.

cc @glemaitre @thomasjpfan

COMMENT:
+0 (can live either way)
A few thoughts: Maybe the fact that we internally use it very often is an indication that should not make it public. Indexing does not seem to be something that scikit-learn should fix, IMO. On the other side, many projects seem to import it even if it's not public.
The difficult part about it is the mix between Array API and Dataframes. The latter alone can be done with narwhals.

Labels: API, Needs Decision
_safe_indexing being the only *private* thing listed in classes.rst

`_safe_indexing` is the only _private_ tool we have listed in `classes.rst`.

`_safe_indexing` was also mentioned in https://github.com/scikit-learn/scikit-learn/issues/15801, but this issue is to have the discussion about this very limited scope.

We should really either remove it from the docs, or make it public.

cc @glemaitre @thomasjpfan

COMMENT:
For the time being, I think we should remove it from the public API, #32728. We can add it again when we conclude to make it public again.

Labels: API, Needs Decision
Enable `drop='constant'` in OneHotEncoder

### Describe the workflow you want to enable

Currently the `drop` parameter in `OneHotEncoder` objects support `{‘first’, ‘if_binary’, <array-of-features>, None}` as potential choices, with a strong encouragement to use `None` to allow proper use of regularized linear modeling downstream. However, if there exists features that are constant for a data matrix then it would be really great if `OneHotEncoder` took care of dropping that feature entirely during coding. If all columns in the input matrix were constant perhaps an error could be raised during `.fit(...)` when this option was used for object initialization.

I would like to do the following.
```
>>> oh_encoder = OneHotEncoder(
    sparse_output=False,
    drop='constant',
    handle_unknown='ignore')
```

So that this were possible.
```
>>> X
array([['M', 'O', 'M'],
       ['M', 'A', 'N']], dtype='<U1')
>>> oh_encoder.fit_transform(X)
array([[0., 1., 0., 1.],
       [1., 0., 1., 0.]], dtype='float')
>>> oh_encoder.inverse_transform(oh_encoder.transform(X))
array([['M', 'O', 'M'],
       ['M', 'A', 'N']], dtype='<U1')
```

Please note that this is distinct from using `drop='first'` which would definitely remove the constant columns but also the first column for each feature group in the data matrix columns, which would be pretty crazy. Perhaps, instead of `None` this could become the default choice as well, as this encoding perfectly preserves all the information in the input data matrix.

### Describe your proposed solution

It would be great to have `drop='constant'` as an option.

### Describe alternatives you've considered, if relevant

I have considered writing my own `OneHotEncoder`, but it feels redudant and error prone to do so. I have also tried to specify the constant features per column via the `drop=<array>` method, but there are columns for which I set values to `None` (corresponding to the non-constant columns) but the object didn't like that.

### Additional context

None.

COMMENT:
I feel that our `OneHotEncoder` already has a lot of parameters. Since the feature being constant is not specific to the `OneHotEncoder`, I am thinking that we should not have the option into it. Indeed, I would prefer to have a specific transformer that only detect constant column seen at `fit` and drop them at `transform`. In this way, you could use this transformer ahead of several other transformation.

Somehow it could also be useful in the context discussed in: https://github.com/scikit-learn/scikit-learn/issues/19547

Labels: New Feature, Needs Decision
Enable `drop='constant'` in OneHotEncoder

### Describe the workflow you want to enable

Currently the `drop` parameter in `OneHotEncoder` objects support `{‘first’, ‘if_binary’, <array-of-features>, None}` as potential choices, with a strong encouragement to use `None` to allow proper use of regularized linear modeling downstream. However, if there exists features that are constant for a data matrix then it would be really great if `OneHotEncoder` took care of dropping that feature entirely during coding. If all columns in the input matrix were constant perhaps an error could be raised during `.fit(...)` when this option was used for object initialization.

I would like to do the following.
```
>>> oh_encoder = OneHotEncoder(
    sparse_output=False,
    drop='constant',
    handle_unknown='ignore')
```

So that this were possible.
```
>>> X
array([['M', 'O', 'M'],
       ['M', 'A', 'N']], dtype='<U1')
>>> oh_encoder.fit_transform(X)
array([[0., 1., 0., 1.],
       [1., 0., 1., 0.]], dtype='float')
>>> oh_encoder.inverse_transform(oh_encoder.transform(X))
array([['M', 'O', 'M'],
       ['M', 'A', 'N']], dtype='<U1')
```

Please note that this is distinct from using `drop='first'` which would definitely remove the constant columns but also the first column for each feature group in the data matrix columns, which would be pretty crazy. Perhaps, instead of `None` this could become the default choice as well, as this encoding perfectly preserves all the information in the input data matrix.

### Describe your proposed solution

It would be great to have `drop='constant'` as an option.

### Describe alternatives you've considered, if relevant

I have considered writing my own `OneHotEncoder`, but it feels redudant and error prone to do so. I have also tried to specify the constant features per column via the `drop=<array>` method, but there are columns for which I set values to `None` (corresponding to the non-constant columns) but the object didn't like that.

### Additional context

None.

COMMENT:
Is there such a transformer available in the library that removes the constant columns and adds them back at the `inverse_transform(...)` time?

Labels: New Feature, Needs Decision
Enable `drop='constant'` in OneHotEncoder

### Describe the workflow you want to enable

Currently the `drop` parameter in `OneHotEncoder` objects support `{‘first’, ‘if_binary’, <array-of-features>, None}` as potential choices, with a strong encouragement to use `None` to allow proper use of regularized linear modeling downstream. However, if there exists features that are constant for a data matrix then it would be really great if `OneHotEncoder` took care of dropping that feature entirely during coding. If all columns in the input matrix were constant perhaps an error could be raised during `.fit(...)` when this option was used for object initialization.

I would like to do the following.
```
>>> oh_encoder = OneHotEncoder(
    sparse_output=False,
    drop='constant',
    handle_unknown='ignore')
```

So that this were possible.
```
>>> X
array([['M', 'O', 'M'],
       ['M', 'A', 'N']], dtype='<U1')
>>> oh_encoder.fit_transform(X)
array([[0., 1., 0., 1.],
       [1., 0., 1., 0.]], dtype='float')
>>> oh_encoder.inverse_transform(oh_encoder.transform(X))
array([['M', 'O', 'M'],
       ['M', 'A', 'N']], dtype='<U1')
```

Please note that this is distinct from using `drop='first'` which would definitely remove the constant columns but also the first column for each feature group in the data matrix columns, which would be pretty crazy. Perhaps, instead of `None` this could become the default choice as well, as this encoding perfectly preserves all the information in the input data matrix.

### Describe your proposed solution

It would be great to have `drop='constant'` as an option.

### Describe alternatives you've considered, if relevant

I have considered writing my own `OneHotEncoder`, but it feels redudant and error prone to do so. I have also tried to specify the constant features per column via the `drop=<array>` method, but there are columns for which I set values to `None` (corresponding to the non-constant columns) but the object didn't like that.

### Additional context

None.

COMMENT:
Part of the reason it would be great to have this within the `OneHotEncoder` object itself is that it automatically would take care of the removed offsets when going for `get_feature_names_out` for example.

Labels: New Feature, Needs Decision
Enable `drop='constant'` in OneHotEncoder

### Describe the workflow you want to enable

Currently the `drop` parameter in `OneHotEncoder` objects support `{‘first’, ‘if_binary’, <array-of-features>, None}` as potential choices, with a strong encouragement to use `None` to allow proper use of regularized linear modeling downstream. However, if there exists features that are constant for a data matrix then it would be really great if `OneHotEncoder` took care of dropping that feature entirely during coding. If all columns in the input matrix were constant perhaps an error could be raised during `.fit(...)` when this option was used for object initialization.

I would like to do the following.
```
>>> oh_encoder = OneHotEncoder(
    sparse_output=False,
    drop='constant',
    handle_unknown='ignore')
```

So that this were possible.
```
>>> X
array([['M', 'O', 'M'],
       ['M', 'A', 'N']], dtype='<U1')
>>> oh_encoder.fit_transform(X)
array([[0., 1., 0., 1.],
       [1., 0., 1., 0.]], dtype='float')
>>> oh_encoder.inverse_transform(oh_encoder.transform(X))
array([['M', 'O', 'M'],
       ['M', 'A', 'N']], dtype='<U1')
```

Please note that this is distinct from using `drop='first'` which would definitely remove the constant columns but also the first column for each feature group in the data matrix columns, which would be pretty crazy. Perhaps, instead of `None` this could become the default choice as well, as this encoding perfectly preserves all the information in the input data matrix.

### Describe your proposed solution

It would be great to have `drop='constant'` as an option.

### Describe alternatives you've considered, if relevant

I have considered writing my own `OneHotEncoder`, but it feels redudant and error prone to do so. I have also tried to specify the constant features per column via the `drop=<array>` method, but there are columns for which I set values to `None` (corresponding to the non-constant columns) but the object didn't like that.

### Additional context

None.

COMMENT:
> Part of the reason it would be great to have this within the OneHotEncoder object itself is that it automatically would take care of the removed offsets when going for get_feature_names_out for example.

Having a pipeline with such a transformer will also propagate the feature names in this manner.

Labels: New Feature, Needs Decision
Enable `drop='constant'` in OneHotEncoder

### Describe the workflow you want to enable

Currently the `drop` parameter in `OneHotEncoder` objects support `{‘first’, ‘if_binary’, <array-of-features>, None}` as potential choices, with a strong encouragement to use `None` to allow proper use of regularized linear modeling downstream. However, if there exists features that are constant for a data matrix then it would be really great if `OneHotEncoder` took care of dropping that feature entirely during coding. If all columns in the input matrix were constant perhaps an error could be raised during `.fit(...)` when this option was used for object initialization.

I would like to do the following.
```
>>> oh_encoder = OneHotEncoder(
    sparse_output=False,
    drop='constant',
    handle_unknown='ignore')
```

So that this were possible.
```
>>> X
array([['M', 'O', 'M'],
       ['M', 'A', 'N']], dtype='<U1')
>>> oh_encoder.fit_transform(X)
array([[0., 1., 0., 1.],
       [1., 0., 1., 0.]], dtype='float')
>>> oh_encoder.inverse_transform(oh_encoder.transform(X))
array([['M', 'O', 'M'],
       ['M', 'A', 'N']], dtype='<U1')
```

Please note that this is distinct from using `drop='first'` which would definitely remove the constant columns but also the first column for each feature group in the data matrix columns, which would be pretty crazy. Perhaps, instead of `None` this could become the default choice as well, as this encoding perfectly preserves all the information in the input data matrix.

### Describe your proposed solution

It would be great to have `drop='constant'` as an option.

### Describe alternatives you've considered, if relevant

I have considered writing my own `OneHotEncoder`, but it feels redudant and error prone to do so. I have also tried to specify the constant features per column via the `drop=<array>` method, but there are columns for which I set values to `None` (corresponding to the non-constant columns) but the object didn't like that.

### Additional context

None.

COMMENT:
I did not think of it at first but you can use the [`VarianceThreshold`](https://scikit-learn.org/stable/modules/generated/sklearn.feature_selection.VarianceThreshold.html) for this purpose.

Labels: New Feature, Needs Decision
`distance_threshold` Behavior with Cosine Metric in AgglomerativeClustering

### Describe the bug

In the documentation for AgglomerativeClustering, the distance_threshold parameter is described as:

> The linkage distance threshold at or above which clusters will not be merged. If not None, n_clusters must be None and compute_full_tree must be True.

If we use the cosine metric, the range of distances is from -1 to 1, with -1 indicating complete dissimilarity and 1 indicating similarity. Using a threshold like `distance_threshold=0.5` under this setting could be ambiguous. Does it mean the algorithm clusters items with distances ranging from -1 to 0.5?

It's common in some contexts to use `1 - cosine_similarity` to get a distance measure that ranges from [0, 2]. If this transformation is applied internally when using the cosine metric, then a `distance_threshold` of 0.5 would have a different interpretation.

Can the documentation provide clarification on:

How the cosine metric is treated internally in terms of distance (e.g., is it transformed to a [0, 2] range)?
How to interpret `distance_threshold` values when using the cosine metric?

Thank you!

### Steps/Code to Reproduce

NA

### Expected Results

Behaviour of distance_threshold with cosine similarity

### Actual Results

Behaviour of distance_threshold with cosine similarity

### Versions

```shell
NA
```

COMMENT:
The cosine similarity is not a distance. When referring to `"cosine"`, we expect the cosine distance definition that is (1 - cosine_similarity): https://scikit-learn.org/stable/modules/generated/sklearn.metrics.pairwise.cosine_distances.html

Labels: Bug, Needs Triage
Implement `make_sparse_spd_matrix` using a sparse memory layout from the start

### Describe the workflow you want to enable

As discussed in #27359, `make_sparse_spd_matrix` actually returns a dense numpy array (with many zero values).

I think it should be possible to rewrite this code to compose operations on sparse arrays/matrices from the start instead of allocating large dense square matrices.

I have not tried myself, but I think it should be doable and it should be much more memory efficient.

COMMENT:
Can I try this one?

Labels: New Feature, Moderate
Implement `make_sparse_spd_matrix` using a sparse memory layout from the start

### Describe the workflow you want to enable

As discussed in #27359, `make_sparse_spd_matrix` actually returns a dense numpy array (with many zero values).

I think it should be possible to rewrite this code to compose operations on sparse arrays/matrices from the start instead of allocating large dense square matrices.

I have not tried myself, but I think it should be doable and it should be much more memory efficient.

COMMENT:
That'd be nice @Charlie-XIAO

Labels: New Feature, Moderate
Implement `make_sparse_spd_matrix` using a sparse memory layout from the start

### Describe the workflow you want to enable

As discussed in #27359, `make_sparse_spd_matrix` actually returns a dense numpy array (with many zero values).

I think it should be possible to rewrite this code to compose operations on sparse arrays/matrices from the start instead of allocating large dense square matrices.

I have not tried myself, but I think it should be doable and it should be much more memory efficient.

COMMENT:
Surprisingly, this function does not seem to be tested at all (if I have not missed anything).

Labels: New Feature, Moderate
Implement `make_sparse_spd_matrix` using a sparse memory layout from the start

### Describe the workflow you want to enable

As discussed in #27359, `make_sparse_spd_matrix` actually returns a dense numpy array (with many zero values).

I think it should be possible to rewrite this code to compose operations on sparse arrays/matrices from the start instead of allocating large dense square matrices.

I have not tried myself, but I think it should be doable and it should be much more memory efficient.

COMMENT:
/take

Labels: New Feature, Moderate
Implement `make_sparse_spd_matrix` using a sparse memory layout from the start

### Describe the workflow you want to enable

As discussed in #27359, `make_sparse_spd_matrix` actually returns a dense numpy array (with many zero values).

I think it should be possible to rewrite this code to compose operations on sparse arrays/matrices from the start instead of allocating large dense square matrices.

I have not tried myself, but I think it should be doable and it should be much more memory efficient.

COMMENT:
> Surprisingly, this function does not seem to be tested at all (if I have not missed anything).

It's implicitly used in `test_graphical_lasso.py` and in `test_extmath.py`, but it should actually be tested in `test_samples_generator.py`.

Labels: New Feature, Moderate
Implement `make_sparse_spd_matrix` using a sparse memory layout from the start

### Describe the workflow you want to enable

As discussed in #27359, `make_sparse_spd_matrix` actually returns a dense numpy array (with many zero values).

I think it should be possible to rewrite this code to compose operations on sparse arrays/matrices from the start instead of allocating large dense square matrices.

I have not tried myself, but I think it should be doable and it should be much more memory efficient.

COMMENT:
True, I'm also seeing those from the CI. It is, however, tricky to test its parameters which I explained in the linked PR.

Labels: New Feature, Moderate
Implement `make_sparse_spd_matrix` using a sparse memory layout from the start

### Describe the workflow you want to enable

As discussed in #27359, `make_sparse_spd_matrix` actually returns a dense numpy array (with many zero values).

I think it should be possible to rewrite this code to compose operations on sparse arrays/matrices from the start instead of allocating large dense square matrices.

I have not tried myself, but I think it should be doable and it should be much more memory efficient.

COMMENT:
Sorry, @Charlie-XIAO, I have overseen that you've already addressed that with the PR.

Labels: New Feature, Moderate
Implement `make_sparse_spd_matrix` using a sparse memory layout from the start

### Describe the workflow you want to enable

As discussed in #27359, `make_sparse_spd_matrix` actually returns a dense numpy array (with many zero values).

I think it should be possible to rewrite this code to compose operations on sparse arrays/matrices from the start instead of allocating large dense square matrices.

I have not tried myself, but I think it should be doable and it should be much more memory efficient.

COMMENT:
Please don't apologize @StefanieSenger, that doesn't matter at all. I just wanted to say that if you are interested, you can look at my analysis in that PR. In fact I have just seen that CI is complaining about `test_extmath.py` when you posted the comment.

Labels: New Feature, Moderate
Implement `make_sparse_spd_matrix` using a sparse memory layout from the start

### Describe the workflow you want to enable

As discussed in #27359, `make_sparse_spd_matrix` actually returns a dense numpy array (with many zero values).

I think it should be possible to rewrite this code to compose operations on sparse arrays/matrices from the start instead of allocating large dense square matrices.

I have not tried myself, but I think it should be doable and it should be much more memory efficient.

COMMENT:
@ogrisel I think we can close this one. Sorry that I forgot to write `Fixes #xxx` in the PR so this was not automatically done.

Labels: New Feature, Moderate
Catching deprecation warnings from examples

Up-to-now, we never check deprecation warnings that are raised when executing our examples.
I assume that we could scrap the RST generated file to find such warning:

```python
from pathlib import Path
from pprint import pprint

from sklearn.utils.fixes import VisibleDeprecationWarning

from conf import sphinx_gallery_conf

warning_types = (DeprecationWarning, FutureWarning, VisibleDeprecationWarning)

auto_examples_path = Path(sphinx_gallery_conf["gallery_dirs"][0])

example_raising_warnings = []
for example_filename in auto_examples_path.rglob("*.rst"):
    with open(example_filename, "r") as f:
        lines = f.readlines()
        for line in lines:
            if any([warning_type.__name__ in line for warning_type in warning_types]):
                example_raising_warnings.append(
                    f"{str(example_filename)}: {line.strip()}"
                )

pprint(example_raising_warnings, width=1_000)
```

What I am wondering is exactly how do we want to use it with the CI. Supposedly, it should be only CircleCI but should target only PRs?

COMMENT:
We should have it in the CI, the same way we have it for pytest/doctest I think. And when in a PR we do `[doc build]`, then these are triggered.

Labels: Build / CI
Catching deprecation warnings from examples

Up-to-now, we never check deprecation warnings that are raised when executing our examples.
I assume that we could scrap the RST generated file to find such warning:

```python
from pathlib import Path
from pprint import pprint

from sklearn.utils.fixes import VisibleDeprecationWarning

from conf import sphinx_gallery_conf

warning_types = (DeprecationWarning, FutureWarning, VisibleDeprecationWarning)

auto_examples_path = Path(sphinx_gallery_conf["gallery_dirs"][0])

example_raising_warnings = []
for example_filename in auto_examples_path.rglob("*.rst"):
    with open(example_filename, "r") as f:
        lines = f.readlines()
        for line in lines:
            if any([warning_type.__name__ in line for warning_type in warning_types]):
                example_raising_warnings.append(
                    f"{str(example_filename)}: {line.strip()}"
                )

pprint(example_raising_warnings, width=1_000)
```

What I am wondering is exactly how do we want to use it with the CI. Supposedly, it should be only CircleCI but should target only PRs?

COMMENT:
> We should have it in the CI, the same way we have it for pytest/doctest I think.

This is where it is tricky. We need to generate the documentation and it is never done on Azure.

On PRs, we can scrap the warnings from the example generated.

However, the only way to properly discover the warning is only when we run `[doc build]` that is almost never done. I would assume that if our nightly CRON job would execute trigger a full documentation build then we could catch those. Otherwise, it is unlikely.

Labels: Build / CI