Talip7/scikit-learn-issues-multilabel huggingface dataset:
ImportError: cannot import name 'InconsistentVersionWarning' in sklearn.exceptions

### Describe the bug

The error message "ImportError: cannot import name 'InconsistentVersionWarning'“ occurs when there is an attempt to import the sklearn

### Steps/Code to Reproduce

import sklearn

### Expected Results

successful import

### Actual Results

ImportError: cannot import name 'InconsistentVersionWarning' from 'sklearn.exceptions' (/path/to/sklearn/exceptions.py)

### Versions

```shell
1.5.2
```

COMMENT:
Thank you for your response. 

This is a somewhat peculiar bug that disappeared after I reopened a virtual environment and reinstall `scikit-learn`, and now I am unable to reproduce it. However, at the time when the import of `sklearn` failed, I located the `InconsistentVersionWarning` in `exception.py` based on the `traceback`, which was referenced on line 18 of `base.py` .

``` python
16 from . import __version__
17 from ._config import config_context, get_config
18 from .exceptions import InconsistentVersionWarning
19 from .utils._estimator_html_repr import _HTMLDocumentationLinkMixin, estimator_html_repr
20 from .utils._metadata_requests import _MetadataRequester, _routing_enabled
```

Upon inspection, I found that `InconsistentVersionWarning` was indeed missing from the `__all__` list in `exception.py`, and after adding it, I was able to import `sklearn` normally.

Labels: Bug, Needs Triage
ImportError: cannot import name 'InconsistentVersionWarning' in sklearn.exceptions

### Describe the bug

The error message "ImportError: cannot import name 'InconsistentVersionWarning'“ occurs when there is an attempt to import the sklearn

### Steps/Code to Reproduce

import sklearn

### Expected Results

successful import

### Actual Results

ImportError: cannot import name 'InconsistentVersionWarning' from 'sklearn.exceptions' (/path/to/sklearn/exceptions.py)

### Versions

```shell
1.5.2
```

COMMENT:
`__all__` does not influence what can be imported from a module only what is imported with wildcard imports e.g. `from sklearn.exceptions import *`, see [this](https://stackoverflow.com/a/64130) for more details.

Honestly it is hard to know what is the root cause in your issue, but I would guess that it is specific to the history of your environment. If you find a way to recreate it consistently, I am curious, but I am not sure there is much we can do in scikit-learn, so I am going to close this one.

Labels: Bug, Needs Triage
proper sparse support in glm's with newton-cholesky

### Describe the workflow you want to enable

When a user fits a glm with a sparse X, I believe the newton-cholesky solver ultimately creates a dense hessian, and the newton step is solved using scipy's dense symmetric linear solve.  Instead I think SKL should create a sparse hessian and use scipy's sparse linear solve.

### Describe your proposed solution

In https://github.com/scikit-learn/scikit-learn/blob/main/sklearn/linear_model/_glm/_newton_solver.py (around line 485) test for sparse X, and then replace sp.linalg.solve with sp.sparse.linalg.spsolve.

I assume there's another place within the _glm codebase which defines the hessian (as in the docstring at the top of https://github.com/scikit-learn/scikit-learn/blob/main/sklearn/linear_model/_glm/_newton_solver.py), but I don't see it.  The docstring suggests that the hessian is created as `H = X.T @ diag(loss.hessian) @ X + l2_reg_strength * identity`.  I'm assuming the `diag` function from numpy is what is used here, and this will cause the resulting H to be dense.  Instead the code would need scipy's sparse.diags() function.

### Describe alternatives you've considered, if relevant

_No response_

### Additional context

_No response_

COMMENT:
First of all, do you encounter any problems/issues with sparse `X` in GLMs?
> the newton-cholesky solver ultimately creates a dense hessian

That‘s correct.
Let’s for a moment forget about the term `diag(loss.hessian)` which is never build explicitly (as dense array). Why do you think `X.T @ X` is sparse for sparse `X`?

Labels: New Feature, module:linear_model
proper sparse support in glm's with newton-cholesky

### Describe the workflow you want to enable

When a user fits a glm with a sparse X, I believe the newton-cholesky solver ultimately creates a dense hessian, and the newton step is solved using scipy's dense symmetric linear solve.  Instead I think SKL should create a sparse hessian and use scipy's sparse linear solve.

### Describe your proposed solution

In https://github.com/scikit-learn/scikit-learn/blob/main/sklearn/linear_model/_glm/_newton_solver.py (around line 485) test for sparse X, and then replace sp.linalg.solve with sp.sparse.linalg.spsolve.

I assume there's another place within the _glm codebase which defines the hessian (as in the docstring at the top of https://github.com/scikit-learn/scikit-learn/blob/main/sklearn/linear_model/_glm/_newton_solver.py), but I don't see it.  The docstring suggests that the hessian is created as `H = X.T @ diag(loss.hessian) @ X + l2_reg_strength * identity`.  I'm assuming the `diag` function from numpy is what is used here, and this will cause the resulting H to be dense.  Instead the code would need scipy's sparse.diags() function.

### Describe alternatives you've considered, if relevant

_No response_

### Additional context

_No response_

COMMENT:
Thanks for your quick response!

You are right that it won’t be generically true, but the use case I have in mind has an X which is dense in for first few columns (call this Xd), and then many columns representing categorical factors in the remaining columns (Xs).  So, if X = [Xd, Xs], and Xs is wide enough, then X’X will be quite sparse.

Labels: New Feature, module:linear_model
proper sparse support in glm's with newton-cholesky

### Describe the workflow you want to enable

When a user fits a glm with a sparse X, I believe the newton-cholesky solver ultimately creates a dense hessian, and the newton step is solved using scipy's dense symmetric linear solve.  Instead I think SKL should create a sparse hessian and use scipy's sparse linear solve.

### Describe your proposed solution

In https://github.com/scikit-learn/scikit-learn/blob/main/sklearn/linear_model/_glm/_newton_solver.py (around line 485) test for sparse X, and then replace sp.linalg.solve with sp.sparse.linalg.spsolve.

I assume there's another place within the _glm codebase which defines the hessian (as in the docstring at the top of https://github.com/scikit-learn/scikit-learn/blob/main/sklearn/linear_model/_glm/_newton_solver.py), but I don't see it.  The docstring suggests that the hessian is created as `H = X.T @ diag(loss.hessian) @ X + l2_reg_strength * identity`.  I'm assuming the `diag` function from numpy is what is used here, and this will cause the resulting H to be dense.  Instead the code would need scipy's sparse.diags() function.

### Describe alternatives you've considered, if relevant

_No response_

### Additional context

_No response_

COMMENT:
I‘ll close a a duplicate of #18893.
Please express your wish of this feature in that issue.

Labels: New Feature, module:linear_model
Docs for estimator types do not list all possible estimator types

### Describe the issue linked to the documentation

The docs for 'Developing scikit-learn estimators' mention that one should specify the estimator type:
https://scikit-learn.org/stable/developers/develop.html#estimator-types

It lists the options as being `"classifier"` and `"regressor"`, but there are more types which scikit-learn uses internally, such as `"outlier_detector"` as used by [OutlierMixin](https://scikit-learn.org/stable/modules/generated/sklearn.base.OutlierMixin.html).

The page for 'Developing scikit-learn estimators' is likely where users will go to browse first, and that one (+ clusterer) is not suggested nor discoverable from the first page.

### Suggest a potential alternative/fix

Should list all of the possible estimator types in the section for "Estimator types"

COMMENT:
Good point, a PR would be more than welcome to improve this!

Edit: clusterer is mentioned but not outlier_detector I think.

> It should be "classifier" for classifiers and "regressor" for regressors and "clusterer" for clustering methods, to work as expected. Inheriting from ClassifierMixin, RegressorMixin or ClusterMixin will set the attribute automatically.

Labels: Easy, Documentation
Docs for estimator types do not list all possible estimator types

### Describe the issue linked to the documentation

The docs for 'Developing scikit-learn estimators' mention that one should specify the estimator type:
https://scikit-learn.org/stable/developers/develop.html#estimator-types

It lists the options as being `"classifier"` and `"regressor"`, but there are more types which scikit-learn uses internally, such as `"outlier_detector"` as used by [OutlierMixin](https://scikit-learn.org/stable/modules/generated/sklearn.base.OutlierMixin.html).

The page for 'Developing scikit-learn estimators' is likely where users will go to browse first, and that one (+ clusterer) is not suggested nor discoverable from the first page.

### Suggest a potential alternative/fix

Should list all of the possible estimator types in the section for "Estimator types"

COMMENT:
/take

Labels: Easy, Documentation
Implications of FrozenEstimator on our API

With https://github.com/scikit-learn/scikit-learn/pull/29705, we have a simple way to freeze estimators, which means there is no need for `cv="prefit"`. This also opens the door for https://github.com/scikit-learn/scikit-learn/pull/8350 to make `Pipeline` and `FeatureUnion` follow our conventions. This issue is to discuss the API implications of introducing `FrozenEstimator`. Here are the two I had in mind:

### cv="prefit"

For the cv case, users pass a frozen estimator directly into cv:

```python
rf = RandomForestClassifer()
rf.fit(X_train, y_train)
frozen_rf = FrozenEstimator(rf)

calibration = CalibratedClassifierCV(frozen_rf)
calibration.fit(X_calib, y_calib)
```

Making this change will simplify our codebase with `cv="prefit"`

### compose.Pipeline

We introduce a new `compose.Pipeline` which follows our conventions with `clone`. (The current `pipeline.Pipeline` does not clone.)

```python
from sklearn.compose import Pipeline

prep = ColumnTransformer(...)
prep.fit(X_train, y_train)
frozen_prep = FrozenEstimator(prep)

pipe = Pipeline([frozen_prep, LogisticRegression()])

pipe.fit(X_another, y_another)
```

---

In both cases, I like prefer the semantics of `FrozenEstimator`.

COMMENT:
Was just talking with @glemaitre about this and we also very much agree with these points (or at least I thought Guillaume agrees 😅 ). I'd like to have the `FrozenEstimator` PR merged, and then open follow up PRs to deal with these implications.

This also deserves a @scikit-learn/core-devs ping.

Labels: API, RFC
Implications of FrozenEstimator on our API

With https://github.com/scikit-learn/scikit-learn/pull/29705, we have a simple way to freeze estimators, which means there is no need for `cv="prefit"`. This also opens the door for https://github.com/scikit-learn/scikit-learn/pull/8350 to make `Pipeline` and `FeatureUnion` follow our conventions. This issue is to discuss the API implications of introducing `FrozenEstimator`. Here are the two I had in mind:

### cv="prefit"

For the cv case, users pass a frozen estimator directly into cv:

```python
rf = RandomForestClassifer()
rf.fit(X_train, y_train)
frozen_rf = FrozenEstimator(rf)

calibration = CalibratedClassifierCV(frozen_rf)
calibration.fit(X_calib, y_calib)
```

Making this change will simplify our codebase with `cv="prefit"`

### compose.Pipeline

We introduce a new `compose.Pipeline` which follows our conventions with `clone`. (The current `pipeline.Pipeline` does not clone.)

```python
from sklearn.compose import Pipeline

prep = ColumnTransformer(...)
prep.fit(X_train, y_train)
frozen_prep = FrozenEstimator(prep)

pipe = Pipeline([frozen_prep, LogisticRegression()])

pipe.fit(X_another, y_another)
```

---

In both cases, I like prefer the semantics of `FrozenEstimator`.

COMMENT:
In addition of `cv="prefit"`, we also have the option `prefit=True` for `SelectFromModel`.

So in general, I think that using the `FrozenEstimator` is the right way because:

- it is explicit which estimator is frozen
- it is less brittle when having nested objects (pipeline in column transformer inside a cross-validation). We never really got it right in the past because this is rather complex to not miss a cloning somewhere.
- it allows to remove a parameter option that is probably of interest of "advanced" user reducing a bit of complexity
- thinking of `cv="prefit"`, the semantic of the parameter & value is not straightforward without reading the documentation (a CV is not really prefitted)
- it simplifies the code internally because `clone` is handled by the `FrozenEstimator` and not anymore in each of the meta-estimator.

To me there is one downside:

- we request to nest an estimator inside a new meta-estimator. However, taken that this is an advanced use-case, I think that this is a mitigated complexity considering the type of user that are going to use the pattern.

So given the pros/cons that I could come with, I'm pretty much in favor of introducing `FrozeEstimator` and deprecate `cv="prefit"` and `prefit=True/False`

Labels: API, RFC
Implications of FrozenEstimator on our API

With https://github.com/scikit-learn/scikit-learn/pull/29705, we have a simple way to freeze estimators, which means there is no need for `cv="prefit"`. This also opens the door for https://github.com/scikit-learn/scikit-learn/pull/8350 to make `Pipeline` and `FeatureUnion` follow our conventions. This issue is to discuss the API implications of introducing `FrozenEstimator`. Here are the two I had in mind:

### cv="prefit"

For the cv case, users pass a frozen estimator directly into cv:

```python
rf = RandomForestClassifer()
rf.fit(X_train, y_train)
frozen_rf = FrozenEstimator(rf)

calibration = CalibratedClassifierCV(frozen_rf)
calibration.fit(X_calib, y_calib)
```

Making this change will simplify our codebase with `cv="prefit"`

### compose.Pipeline

We introduce a new `compose.Pipeline` which follows our conventions with `clone`. (The current `pipeline.Pipeline` does not clone.)

```python
from sklearn.compose import Pipeline

prep = ColumnTransformer(...)
prep.fit(X_train, y_train)
frozen_prep = FrozenEstimator(prep)

pipe = Pipeline([frozen_prep, LogisticRegression()])

pipe.fit(X_another, y_another)
```

---

In both cases, I like prefer the semantics of `FrozenEstimator`.

COMMENT:
> To me there is one downside:
> 
> * we request to nest an estimator inside a new meta-estimator. However, taken that this is an advanced use-case, I think that this is a mitigated complexity considering the type of user that are going to use the pattern.
> 
> So given the pros/cons that I could come with, I'm pretty much in favor of introducing `FrozeEstimator` and deprecate `cv="prefit"` and `prefit=True/False`

Would it be weird to add a `freeze() / get_frozen()` method to `BaseEstimator`, so that all estimators can effectively return a `FrozenEstimator` without having to import `FrozenEstimator` and do the nesting?

Perhaps it adds too much complexity and the user is confused because then there are multiple ways of "getting a frozen estimator"?

Labels: API, RFC
Implications of FrozenEstimator on our API

With https://github.com/scikit-learn/scikit-learn/pull/29705, we have a simple way to freeze estimators, which means there is no need for `cv="prefit"`. This also opens the door for https://github.com/scikit-learn/scikit-learn/pull/8350 to make `Pipeline` and `FeatureUnion` follow our conventions. This issue is to discuss the API implications of introducing `FrozenEstimator`. Here are the two I had in mind:

### cv="prefit"

For the cv case, users pass a frozen estimator directly into cv:

```python
rf = RandomForestClassifer()
rf.fit(X_train, y_train)
frozen_rf = FrozenEstimator(rf)

calibration = CalibratedClassifierCV(frozen_rf)
calibration.fit(X_calib, y_calib)
```

Making this change will simplify our codebase with `cv="prefit"`

### compose.Pipeline

We introduce a new `compose.Pipeline` which follows our conventions with `clone`. (The current `pipeline.Pipeline` does not clone.)

```python
from sklearn.compose import Pipeline

prep = ColumnTransformer(...)
prep.fit(X_train, y_train)
frozen_prep = FrozenEstimator(prep)

pipe = Pipeline([frozen_prep, LogisticRegression()])

pipe.fit(X_another, y_another)
```

---

In both cases, I like prefer the semantics of `FrozenEstimator`.

COMMENT:
I wouldn't mind a `def get_frozen(self): return FrozenEstimator(self)` indeed. But I think we can move forward with the rest, and decide if we want to add this nicety later. Good suggestion though.

Labels: API, RFC
Implications of FrozenEstimator on our API

With https://github.com/scikit-learn/scikit-learn/pull/29705, we have a simple way to freeze estimators, which means there is no need for `cv="prefit"`. This also opens the door for https://github.com/scikit-learn/scikit-learn/pull/8350 to make `Pipeline` and `FeatureUnion` follow our conventions. This issue is to discuss the API implications of introducing `FrozenEstimator`. Here are the two I had in mind:

### cv="prefit"

For the cv case, users pass a frozen estimator directly into cv:

```python
rf = RandomForestClassifer()
rf.fit(X_train, y_train)
frozen_rf = FrozenEstimator(rf)

calibration = CalibratedClassifierCV(frozen_rf)
calibration.fit(X_calib, y_calib)
```

Making this change will simplify our codebase with `cv="prefit"`

### compose.Pipeline

We introduce a new `compose.Pipeline` which follows our conventions with `clone`. (The current `pipeline.Pipeline` does not clone.)

```python
from sklearn.compose import Pipeline

prep = ColumnTransformer(...)
prep.fit(X_train, y_train)
frozen_prep = FrozenEstimator(prep)

pipe = Pipeline([frozen_prep, LogisticRegression()])

pipe.fit(X_another, y_another)
```

---

In both cases, I like prefer the semantics of `FrozenEstimator`.

COMMENT:
@ogrisel you might like the `get_frozen()` idea here if you don't want users to have to do an extra import.

Labels: API, RFC
Implications of FrozenEstimator on our API

With https://github.com/scikit-learn/scikit-learn/pull/29705, we have a simple way to freeze estimators, which means there is no need for `cv="prefit"`. This also opens the door for https://github.com/scikit-learn/scikit-learn/pull/8350 to make `Pipeline` and `FeatureUnion` follow our conventions. This issue is to discuss the API implications of introducing `FrozenEstimator`. Here are the two I had in mind:

### cv="prefit"

For the cv case, users pass a frozen estimator directly into cv:

```python
rf = RandomForestClassifer()
rf.fit(X_train, y_train)
frozen_rf = FrozenEstimator(rf)

calibration = CalibratedClassifierCV(frozen_rf)
calibration.fit(X_calib, y_calib)
```

Making this change will simplify our codebase with `cv="prefit"`

### compose.Pipeline

We introduce a new `compose.Pipeline` which follows our conventions with `clone`. (The current `pipeline.Pipeline` does not clone.)

```python
from sklearn.compose import Pipeline

prep = ColumnTransformer(...)
prep.fit(X_train, y_train)
frozen_prep = FrozenEstimator(prep)

pipe = Pipeline([frozen_prep, LogisticRegression()])

pipe.fit(X_another, y_another)
```

---

In both cases, I like prefer the semantics of `FrozenEstimator`.

COMMENT:
> I wouldn't mind a def get_frozen(self): return FrozenEstimator(self) indeed.

No strong opinion on whether we need it or not, but if we introduce it, I would vote in favor of a top level `sklearn.freeze` to be consistent with our existing `sklearn.clone`.

Labels: API, RFC
Implications of FrozenEstimator on our API

With https://github.com/scikit-learn/scikit-learn/pull/29705, we have a simple way to freeze estimators, which means there is no need for `cv="prefit"`. This also opens the door for https://github.com/scikit-learn/scikit-learn/pull/8350 to make `Pipeline` and `FeatureUnion` follow our conventions. This issue is to discuss the API implications of introducing `FrozenEstimator`. Here are the two I had in mind:

### cv="prefit"

For the cv case, users pass a frozen estimator directly into cv:

```python
rf = RandomForestClassifer()
rf.fit(X_train, y_train)
frozen_rf = FrozenEstimator(rf)

calibration = CalibratedClassifierCV(frozen_rf)
calibration.fit(X_calib, y_calib)
```

Making this change will simplify our codebase with `cv="prefit"`

### compose.Pipeline

We introduce a new `compose.Pipeline` which follows our conventions with `clone`. (The current `pipeline.Pipeline` does not clone.)

```python
from sklearn.compose import Pipeline

prep = ColumnTransformer(...)
prep.fit(X_train, y_train)
frozen_prep = FrozenEstimator(prep)

pipe = Pipeline([frozen_prep, LogisticRegression()])

pipe.fit(X_another, y_another)
```

---

In both cases, I like prefer the semantics of `FrozenEstimator`.

COMMENT:
> @ogrisel you might like the get_frozen() idea here if you don't want users to have to do an extra import.

You still need to import it (unless you already imported the top level `sklearn` for other things).

EDIT: this comment is wrong.

Labels: API, RFC
Implications of FrozenEstimator on our API

With https://github.com/scikit-learn/scikit-learn/pull/29705, we have a simple way to freeze estimators, which means there is no need for `cv="prefit"`. This also opens the door for https://github.com/scikit-learn/scikit-learn/pull/8350 to make `Pipeline` and `FeatureUnion` follow our conventions. This issue is to discuss the API implications of introducing `FrozenEstimator`. Here are the two I had in mind:

### cv="prefit"

For the cv case, users pass a frozen estimator directly into cv:

```python
rf = RandomForestClassifer()
rf.fit(X_train, y_train)
frozen_rf = FrozenEstimator(rf)

calibration = CalibratedClassifierCV(frozen_rf)
calibration.fit(X_calib, y_calib)
```

Making this change will simplify our codebase with `cv="prefit"`

### compose.Pipeline

We introduce a new `compose.Pipeline` which follows our conventions with `clone`. (The current `pipeline.Pipeline` does not clone.)

```python
from sklearn.compose import Pipeline

prep = ColumnTransformer(...)
prep.fit(X_train, y_train)
frozen_prep = FrozenEstimator(prep)

pipe = Pipeline([frozen_prep, LogisticRegression()])

pipe.fit(X_another, y_another)
```

---

In both cases, I like prefer the semantics of `FrozenEstimator`.

COMMENT:
Let's finalize the reviews of  #30171 and  #30172 first, and then let's open a PR to explore how `sklearn.freeze` (or `sklearn.frozen.get_frozen`) would feel in the examples.

Labels: API, RFC