Talip7/scikit-learn-issues-multilabel huggingface dataset:
RFC Supporting `scipy.sparse.sparray`

### Context

SciPy is now favoring sparse arrays (i.e. `scipy.sparse.sparray` and its subclasses) over sparse matrices (i.e. `scipy.sparse.spmatrix` and its subclasses) to enlarge the scope of matrices to $n$-dimensional data-structures since SciPy 1.8 (see https://github.com/scipy/scipy/pull/14822).

Sparse matrices now subclass their associated sparse arrays (see https://github.com/scipy/scipy/pull/18440).

scikit-learn has been supporting sparse matrices but now also needs to support SciPy sparse arrays.

### Proposed solutions

Ordered by preference:
 - Use `scipy.sparse.issparse` and the `format` attribute everywhere and not use `isinstance` at all
 - Use `isinstance` on private compatibility class defined to be `scipy.sparse.spmatrix` or `scipy.sparse.sparray` conditionally on SciPy's version (i.e. use `scipy.sparse.sparray` if available)
 - Rely on duck-typing or the class name to check for sparse arrays

cc @ivirshup

COMMENT:
Yes, the conversion between spmatrix and sparray is zero copy (unless you use the kwarg `copy=True`).

So, I think you are suggesting that we could use sparray or spmatrix internally and convert to `get_config()["sparse_format"]` at the end of any function that returns sparse. And similarly, estimator attributes that are sparse can be constructed internally using either sparry or spmatrix and converted when assigned to the estimator.

That could mean if they input spmatrix while config is set to sparray, they might input spmatrix and get back sparray. (which seems fine so long as the config setting is widely known and understood). Do I understand correctly?

In terms of implementation, we could make a PR that implements the config option and an idiom for performing the conversion just before storing/returning any sparse result. Should an idiom like this get put into a helper function, e.g. `X = convert_sparse_to_sparse_format(X)` with a better name)? Where would that helper function live?

I would then want to make a PR to convert the internal creation of sparse to use the sparray pattern. This would set up sklearn to require minimal changes as the rest of the migration to sparray occurs. hopefully only changes to the idiom, and in the very distant future, removal of the call to the helper.

Here's a proposed idiom, just to make sure I am understanding how the config idea would work: 
```python
    if sparse.issparse(X):
        sparse_format = get_config()["sparse_format"]
        if not isinstance(X, sparse_format):
            klass = X.format + '_' + sparse_format[2:]
            X = sparse.getattr(klass)(X)
```

Labels: New Feature, API, Needs Decision, RFC
RFC Supporting `scipy.sparse.sparray`

### Context

SciPy is now favoring sparse arrays (i.e. `scipy.sparse.sparray` and its subclasses) over sparse matrices (i.e. `scipy.sparse.spmatrix` and its subclasses) to enlarge the scope of matrices to $n$-dimensional data-structures since SciPy 1.8 (see https://github.com/scipy/scipy/pull/14822).

Sparse matrices now subclass their associated sparse arrays (see https://github.com/scipy/scipy/pull/18440).

scikit-learn has been supporting sparse matrices but now also needs to support SciPy sparse arrays.

### Proposed solutions

Ordered by preference:
 - Use `scipy.sparse.issparse` and the `format` attribute everywhere and not use `isinstance` at all
 - Use `isinstance` on private compatibility class defined to be `scipy.sparse.spmatrix` or `scipy.sparse.sparray` conditionally on SciPy's version (i.e. use `scipy.sparse.sparray` if available)
 - Rely on duck-typing or the class name to check for sparse arrays

cc @ivirshup

COMMENT:
> That could mean if they input spmatrix while config is set to sparray, they might input spmatrix and get back sparray. (which seems fine so long as the config setting is widely known and understood). Do I understand correctly?

Yes, that is correct. We have the same semantics for `set_config(transform_output="pandas")`, where the output type is independent of the input type.

> Where would that helper function live?

I'll say a new `sklearn.utils._sparse` module. We may need to make it a public module for third party estimator developers to have the same semantics as scikit-learn.

As for implementation, I'm thinking of having a single function that changes the format based on the global config:

```python
def _as_sparse(X_sparse):
    # check that X_sparse is sparse (if not raise an error)
    ...

    # assume there are only two sparse formats
    global_is_sparray =  get_config()["sparse_format"] == "sparray"
    X_is_sparray = _is_sparray(X_sparse)

    if global_is_sparray:
        if X_is_sparray:
            return X_sparse
        return _convert_from_spmatrix_to_sparray(X_sparse)
    else:  # global is spmatrix
        if not X_is_sparray:
            return X_sparse
        return _convert_from_sparray_to_spmatrix(X_sparse)
```

And then call it everywhere we expose a sparse object in the public API.

Labels: New Feature, API, Needs Decision, RFC
RFC Supporting `scipy.sparse.sparray`

### Context

SciPy is now favoring sparse arrays (i.e. `scipy.sparse.sparray` and its subclasses) over sparse matrices (i.e. `scipy.sparse.spmatrix` and its subclasses) to enlarge the scope of matrices to $n$-dimensional data-structures since SciPy 1.8 (see https://github.com/scipy/scipy/pull/14822).

Sparse matrices now subclass their associated sparse arrays (see https://github.com/scipy/scipy/pull/18440).

scikit-learn has been supporting sparse matrices but now also needs to support SciPy sparse arrays.

### Proposed solutions

Ordered by preference:
 - Use `scipy.sparse.issparse` and the `format` attribute everywhere and not use `isinstance` at all
 - Use `isinstance` on private compatibility class defined to be `scipy.sparse.spmatrix` or `scipy.sparse.sparray` conditionally on SciPy's version (i.e. use `scipy.sparse.sparray` if available)
 - Rely on duck-typing or the class name to check for sparse arrays

cc @ivirshup

COMMENT:
@thomasjpfan I would very much like to avoid to introduce a global config option for sparse array/matrix. I would rather consider moving parallel to deprecation in scipy and make the behaviour dependent on the scipy version, e.g. output sparse array as soon as scipy deprecates it.

Could we list entries in the public API that are affected?

Labels: New Feature, API, Needs Decision, RFC
RFC Supporting `scipy.sparse.sparray`

### Context

SciPy is now favoring sparse arrays (i.e. `scipy.sparse.sparray` and its subclasses) over sparse matrices (i.e. `scipy.sparse.spmatrix` and its subclasses) to enlarge the scope of matrices to $n$-dimensional data-structures since SciPy 1.8 (see https://github.com/scipy/scipy/pull/14822).

Sparse matrices now subclass their associated sparse arrays (see https://github.com/scipy/scipy/pull/18440).

scikit-learn has been supporting sparse matrices but now also needs to support SciPy sparse arrays.

### Proposed solutions

Ordered by preference:
 - Use `scipy.sparse.issparse` and the `format` attribute everywhere and not use `isinstance` at all
 - Use `isinstance` on private compatibility class defined to be `scipy.sparse.spmatrix` or `scipy.sparse.sparray` conditionally on SciPy's version (i.e. use `scipy.sparse.sparray` if available)
 - Rely on duck-typing or the class name to check for sparse arrays

cc @ivirshup

COMMENT:
I would also like to avoid introducing a global config for that. Can't we just add an entry in the changelog of some future release that all sparse outputs will be sparse arrays 2 releases later (or longer if we think it's needed) ?
I think that it can even be done in 1.7 since our scipy min version supports sparse arrays.

Labels: New Feature, API, Needs Decision, RFC
RFC Supporting `scipy.sparse.sparray`

### Context

SciPy is now favoring sparse arrays (i.e. `scipy.sparse.sparray` and its subclasses) over sparse matrices (i.e. `scipy.sparse.spmatrix` and its subclasses) to enlarge the scope of matrices to $n$-dimensional data-structures since SciPy 1.8 (see https://github.com/scipy/scipy/pull/14822).

Sparse matrices now subclass their associated sparse arrays (see https://github.com/scipy/scipy/pull/18440).

scikit-learn has been supporting sparse matrices but now also needs to support SciPy sparse arrays.

### Proposed solutions

Ordered by preference:
 - Use `scipy.sparse.issparse` and the `format` attribute everywhere and not use `isinstance` at all
 - Use `isinstance` on private compatibility class defined to be `scipy.sparse.spmatrix` or `scipy.sparse.sparray` conditionally on SciPy's version (i.e. use `scipy.sparse.sparray` if available)
 - Rely on duck-typing or the class name to check for sparse arrays

cc @ivirshup

COMMENT:
Let's say we deprecate sparse matrix now and in 3 versions we switch to sparse arrays everywhere. What can a user do now to prepare for the transition while scikit-learn is still returning sparse matrices? I would want a way to configure scikit-learn to output sparse arrays and make sure my code works.

Labels: New Feature, API, Needs Decision, RFC
RFC Supporting `scipy.sparse.sparray`

### Context

SciPy is now favoring sparse arrays (i.e. `scipy.sparse.sparray` and its subclasses) over sparse matrices (i.e. `scipy.sparse.spmatrix` and its subclasses) to enlarge the scope of matrices to $n$-dimensional data-structures since SciPy 1.8 (see https://github.com/scipy/scipy/pull/14822).

Sparse matrices now subclass their associated sparse arrays (see https://github.com/scipy/scipy/pull/18440).

scikit-learn has been supporting sparse matrices but now also needs to support SciPy sparse arrays.

### Proposed solutions

Ordered by preference:
 - Use `scipy.sparse.issparse` and the `format` attribute everywhere and not use `isinstance` at all
 - Use `isinstance` on private compatibility class defined to be `scipy.sparse.spmatrix` or `scipy.sparse.sparray` conditionally on SciPy's version (i.e. use `scipy.sparse.sparray` if available)
 - Rely on duck-typing or the class name to check for sparse arrays

cc @ivirshup

COMMENT:
Fair point. My concern is that

> When sparse_format="sparray" everything that scikit-learn produces that is sparse will be a sparray. i.e function returns, estimator attributes, estimator properties, etc.

can represent a ton of places to adapt and test.

Maybe this is a motivation for scikit-learn 2 ?

Labels: New Feature, API, Needs Decision, RFC
RFC Supporting `scipy.sparse.sparray`

### Context

SciPy is now favoring sparse arrays (i.e. `scipy.sparse.sparray` and its subclasses) over sparse matrices (i.e. `scipy.sparse.spmatrix` and its subclasses) to enlarge the scope of matrices to $n$-dimensional data-structures since SciPy 1.8 (see https://github.com/scipy/scipy/pull/14822).

Sparse matrices now subclass their associated sparse arrays (see https://github.com/scipy/scipy/pull/18440).

scikit-learn has been supporting sparse matrices but now also needs to support SciPy sparse arrays.

### Proposed solutions

Ordered by preference:
 - Use `scipy.sparse.issparse` and the `format` attribute everywhere and not use `isinstance` at all
 - Use `isinstance` on private compatibility class defined to be `scipy.sparse.spmatrix` or `scipy.sparse.sparray` conditionally on SciPy's version (i.e. use `scipy.sparse.sparray` if available)
 - Rely on duck-typing or the class name to check for sparse arrays

cc @ivirshup

COMMENT:
I don't think our behavior should be dependent on user's installed scipy version. It would be really hard to pinpoint the problem if user has not changed scikit-learn version but found that the behavior changed.

I also think that global config is not the most desirable option here. For me the most intuitive way is that `f(X: *_array, ...) -> *_array` and `f(X: *_matrix, ...) -> *_matrix` as mentioned in https://github.com/scikit-learn/scikit-learn/issues/26418#issuecomment-1683845888 so that scikit-learn automatically reacts to user's migration at fine granularity. Suppose that there is a global configuration, it would be inconvenient if user only wants to migrate part of the code to use sparray (unless with some trick), and again it seems a bit counter-intuitive to me if I input spmatrix but get back sparray or vice versa, even if I'm aware of that global config.

However there are indeed functions/methods that do not take sparse input but produces sparse output. If there are not too many of them, probably its feasible to add a temporary per-function/method parameter that controls the behavior and follow the normal deprecation cycle. Otherwise probably we would have to have a global config (but IMO it should not affect functions/methods that does take sparse input).

Labels: New Feature, API, Needs Decision, RFC
RFC Supporting `scipy.sparse.sparray`

### Context

SciPy is now favoring sparse arrays (i.e. `scipy.sparse.sparray` and its subclasses) over sparse matrices (i.e. `scipy.sparse.spmatrix` and its subclasses) to enlarge the scope of matrices to $n$-dimensional data-structures since SciPy 1.8 (see https://github.com/scipy/scipy/pull/14822).

Sparse matrices now subclass their associated sparse arrays (see https://github.com/scipy/scipy/pull/18440).

scikit-learn has been supporting sparse matrices but now also needs to support SciPy sparse arrays.

### Proposed solutions

Ordered by preference:
 - Use `scipy.sparse.issparse` and the `format` attribute everywhere and not use `isinstance` at all
 - Use `isinstance` on private compatibility class defined to be `scipy.sparse.spmatrix` or `scipy.sparse.sparray` conditionally on SciPy's version (i.e. use `scipy.sparse.sparray` if available)
 - Rely on duck-typing or the class name to check for sparse arrays

cc @ivirshup

COMMENT:
There are 2 different use cases to consider:
1. `transform(X)` and `predict(X)` return a sparse object, sometimes triggered by the estimator parameter `sparse_output`.
2. Attributes of estimators like `lasso_model.sparse_coef_`.

Labels: New Feature, API, Needs Decision, RFC
RFC Supporting `scipy.sparse.sparray`

### Context

SciPy is now favoring sparse arrays (i.e. `scipy.sparse.sparray` and its subclasses) over sparse matrices (i.e. `scipy.sparse.spmatrix` and its subclasses) to enlarge the scope of matrices to $n$-dimensional data-structures since SciPy 1.8 (see https://github.com/scipy/scipy/pull/14822).

Sparse matrices now subclass their associated sparse arrays (see https://github.com/scipy/scipy/pull/18440).

scikit-learn has been supporting sparse matrices but now also needs to support SciPy sparse arrays.

### Proposed solutions

Ordered by preference:
 - Use `scipy.sparse.issparse` and the `format` attribute everywhere and not use `isinstance` at all
 - Use `isinstance` on private compatibility class defined to be `scipy.sparse.spmatrix` or `scipy.sparse.sparray` conditionally on SciPy's version (i.e. use `scipy.sparse.sparray` if available)
 - Rely on duck-typing or the class name to check for sparse arrays

cc @ivirshup

COMMENT:
It's probably good to keep in mind that users who avoid `*` and instead use `@` and `.multiply()` will likely not care which sparse interface they get.

I'm seeing two approaches proposed in these comments and they both have concrete example implementations (for only part of the library) in existing PRs. Take a look if you think a concrete example is helpful for your decision making.
- a global config `sparse_interface` parameter to use library wide: 
    #31177
- a per-function-call approach using an input to indicate return type:
   #31072  

The places in the code that this affects are laid out in [an above comment](https://github.com/scikit-learn/scikit-learn/issues/26418#issuecomment-2727775191).  To summarize:
- there are common places like `.fit()` and `.transform()` methods 
- some functions can return sparse based on a `bool` keyword that could be made `bool or string`.
- there is the `sparse_coef` property 
- there are four other "misfit functions", listed in that linked comment above, with no existing way to indicate the sparse interface.

I hope this helps focus the discussion on whether to use a library wide parameter or a function-by-function approach.   Thanks!

Labels: New Feature, API, Needs Decision, RFC
RFC Supporting `scipy.sparse.sparray`

### Context

SciPy is now favoring sparse arrays (i.e. `scipy.sparse.sparray` and its subclasses) over sparse matrices (i.e. `scipy.sparse.spmatrix` and its subclasses) to enlarge the scope of matrices to $n$-dimensional data-structures since SciPy 1.8 (see https://github.com/scipy/scipy/pull/14822).

Sparse matrices now subclass their associated sparse arrays (see https://github.com/scipy/scipy/pull/18440).

scikit-learn has been supporting sparse matrices but now also needs to support SciPy sparse arrays.

### Proposed solutions

Ordered by preference:
 - Use `scipy.sparse.issparse` and the `format` attribute everywhere and not use `isinstance` at all
 - Use `isinstance` on private compatibility class defined to be `scipy.sparse.spmatrix` or `scipy.sparse.sparray` conditionally on SciPy's version (i.e. use `scipy.sparse.sparray` if available)
 - Rely on duck-typing or the class name to check for sparse arrays

cc @ivirshup

COMMENT:
In the dev meeting 28 April 2025, we decided to go with a global config. Which options it has and further details must be worked out.

Labels: New Feature, API, Needs Decision, RFC
Calculation of splitting criteria to be executed in parallel threads

### Describe the workflow you want to enable

Currently the computation of splitting criteria for decision tree is single threaded, in theory this can be computed in parallel depending on features available, can the existing package be enhanced to allow multithreading/multiprocessing. 

### Describe your proposed solution

A parameter to existing package where user can specify how many threads should be used to compute splitting criteria for decision tree. Even better, the package is enhanced to understand how many threads are available, how many features are required for splitting criteria at each step and the calculate threads dynamically and then compute splitting in parallel. 

### Describe alternatives you've considered, if relevant

_No response_

### Additional context

_No response_

COMMENT:
Thank you for opening this issue! The most common use case for the decision trees are in the random forest, which is already multi-threaded. To avoid over subscription in random forest, we'll have to make sure that individual trees are single threaded when random forest is multi-threaded.

Given the above common use case, I do not think adding multi-threading to trees is worth the maintenance cost and added complexity. Although I see the benefit of multi-threading the individual trees, I am -1 on the feature proposal.

Labels: New Feature, module:tree, Needs Decision - Include Feature
Calculation of splitting criteria to be executed in parallel threads

### Describe the workflow you want to enable

Currently the computation of splitting criteria for decision tree is single threaded, in theory this can be computed in parallel depending on features available, can the existing package be enhanced to allow multithreading/multiprocessing. 

### Describe your proposed solution

A parameter to existing package where user can specify how many threads should be used to compute splitting criteria for decision tree. Even better, the package is enhanced to understand how many threads are available, how many features are required for splitting criteria at each step and the calculate threads dynamically and then compute splitting in parallel. 

### Describe alternatives you've considered, if relevant

_No response_

### Additional context

_No response_

COMMENT:
I agree, decision trees are almost always used in the context of an ensemble, which are parallelized.

Labels: New Feature, module:tree, Needs Decision - Include Feature
Ability to specify depth of a decision tree

### Describe the workflow you want to enable

There is no parameter currently in decision tree package (both regression and classification) to specify the depth of a tree. There are parameters to limit the number of nodes (maximum nodes) and minimum number of leaves for each node but there is no parameter to specify minimum number of nodes I want in the tree

### Describe your proposed solution

I am not expert in object oriented programming but you could grow the tree as deep as having two points for each leaf node.
If you add a parameter to specify minimum number of leaf nodes for a tree, it gives control to user to grow tree.

### Describe alternatives you've considered, if relevant

_No response_

### Additional context

_No response_

COMMENT:
The parameter is called `max_depth`. But if the tree already achieves other criteria, that max depth is not reached, however, you can control it with that parameter.

Labels: New Feature, Needs Triage
Ability to specify depth of a decision tree

### Describe the workflow you want to enable

There is no parameter currently in decision tree package (both regression and classification) to specify the depth of a tree. There are parameters to limit the number of nodes (maximum nodes) and minimum number of leaves for each node but there is no parameter to specify minimum number of nodes I want in the tree

### Describe your proposed solution

I am not expert in object oriented programming but you could grow the tree as deep as having two points for each leaf node.
If you add a parameter to specify minimum number of leaf nodes for a tree, it gives control to user to grow tree.

### Describe alternatives you've considered, if relevant

_No response_

### Additional context

_No response_

COMMENT:
I need min_depth. In other words, I want to grow the tree more than what it should. This feature is not for actual machine learning but for my research. So even if decision tree is able to build a model with depth 7, I want the tree to grow even further and overfit.
If this feature is not possible for general use, can someone help me code it and make it available in private repo.

Labels: New Feature, Needs Triage
⚠️ CI failed on linux_arm64_wheel ⚠️

**CI failed on [linux_arm64_wheel](https://cirrus-ci.com/build/5008160381992960)** (May 22, 2023)

COMMENT:
## CI is no longer failing! ✅

[Successful run](https://cirrus-ci.com/build/4662728275525632) on May 23, 2023

Labels: Needs Triage
NameError                                 Traceback (most recent call last) Cell In[23], line 1 ----> 1 sc_X = StandardScaler()       2 X =  pd.DataFrame(sc_X.fit_transform(diabetes_df_copy.drop(["Outcome"],axis = 1),), columns=['Pregnancies',        3 'Glucose', 'BloodPressure', 'SkinThickness', 'Insulin', 'BMI', 'DiabetesPedigreeFunction', 'Age'])       4 X.head()  NameError: name 'StandardScaler' is not defined

COMMENT:
Spam

Labels: Needs Triage
⚠️ CI failed on linux_arm64_wheel ⚠️

**CI failed on [linux_arm64_wheel](https://cirrus-ci.com/build/5939226378764288)** (May 20, 2023)

COMMENT:
## CI is no longer failing! ✅

[Successful run](https://cirrus-ci.com/build/4817068697059328) on May 21, 2023

Labels: Needs Triage
SGDClassifier should use sparse representation for coefficients

### Describe the workflow you want to enable

`SGDClassifier` should take a `sparse_coef` initial parameter (default false).  If set to `True`, the initial `coef` will be sparse.  For models like `HashVectorizer` (default 1M features), having a dense coefficient is very inefficient.

### Describe your proposed solution

See above.

### Describe alternatives you've considered, if relevant

Automatically infer `coef_`'s sparsity based on input's spasity!

### Additional context

_No response_

COMMENT:
It will only make sense if you are using L1 penalty, other than that it won't matter much

Labels: New Feature, Needs Triage