Talip7/scikit-learn-issues-multilabel huggingface dataset:
ValueError: The covariance matrix of the support data is equal to 0 - Elliptic Envelope

### Describe the bug

I have been using Elliptic Envelope on a simple time series dataset with default parameters and only setting `contamination` value to `0.5`. However it throws me error like below:

`ValueError: The covariance matrix of the support data is equal to 0, try to increase support_fraction`

Looking at the error, I referred to the documentation and the equation of `support_fraction` does not seem to be clear. It sayd ->  `[n_sample + n_features + 1] / 2`. `Range is (0, 1)`. The value of `n_sample` and `n_features` can never be such that it will fall within 0 and 1. Could you please help in understanding and how this issue can be resolved?

### Steps/Code to Reproduce

```py
model = EllipticEnvelope(contamination=0.5)
model.fit(X)
```

### Expected Results

No error is thrown

### Actual Results

```
File /opt/anaconda3/envs/benchmark/lib/python3.9/site-packages/sklearn/covariance/_elliptic_envelope.py:145, in EllipticEnvelope.fit(self, X, y)
    134 def fit(self, X, y=None):
    135     """Fit the EllipticEnvelope model.
    136 
    137     Parameters
   (...)
    143         Not used, present for API consistency by convention.
    144     """
--> 145     super().fit(X)
    146     self.offset_ = np.percentile(-self.dist_, 100. * self.contamination)
    147     return self

File /opt/anaconda3/envs/benchmark/lib/python3.9/site-packages/sklearn/covariance/_robust_covariance.py:668, in MinCovDet.fit(self, X, y)
    666 self.dist_ = raw_dist
    667 # obtain consistency at normal models
--> 668 self.correct_covariance(X)
    669 # re-weight estimator
    670 self.reweight_covariance(X)

File /opt/anaconda3/envs/benchmark/lib/python3.9/site-packages/sklearn/covariance/_robust_covariance.py:705, in MinCovDet.correct_covariance(self, data)
    703 n_support = np.sum(self.support_)
    704 if n_support < n_samples and np.allclose(self.raw_covariance_, 0):
--> 705     raise ValueError('The covariance matrix of the support data '
    706                      'is equal to 0, try to increase support_fraction')
    707 correction = np.median(self.dist_) / chi2(data.shape[1]).isf(0.5)
    708 covariance_corrected = self.raw_covariance_ * correction

ValueError: The covariance matrix of the support data is equal to 0, try to increase support_fraction
```

### Versions

```shell
0.24.1
```

COMMENT:
Here are the steps to reproduce it:
```
X = ([[1.04725631],
       [1.84803131],
       [1.29789207],
       [1.59315107],
       [0.79657553],
       [0.        ],
       [0.        ],
       [0.        ],
       [0.        ],
       [0.89970052],
       [0.        ],
       [0.        ],
       [0.        ],
       [0.        ],
       [0.        ],
       [0.        ],
       [0.        ],
       [0.        ],
       [0.        ],
       [0.        ]])
model = EllipticEnvelope(contamination=0.0003)
model.fit(X)
```

Labels: Bug, Needs Info
ValueError: The covariance matrix of the support data is equal to 0 - Elliptic Envelope

### Describe the bug

I have been using Elliptic Envelope on a simple time series dataset with default parameters and only setting `contamination` value to `0.5`. However it throws me error like below:

`ValueError: The covariance matrix of the support data is equal to 0, try to increase support_fraction`

Looking at the error, I referred to the documentation and the equation of `support_fraction` does not seem to be clear. It sayd ->  `[n_sample + n_features + 1] / 2`. `Range is (0, 1)`. The value of `n_sample` and `n_features` can never be such that it will fall within 0 and 1. Could you please help in understanding and how this issue can be resolved?

### Steps/Code to Reproduce

```py
model = EllipticEnvelope(contamination=0.5)
model.fit(X)
```

### Expected Results

No error is thrown

### Actual Results

```
File /opt/anaconda3/envs/benchmark/lib/python3.9/site-packages/sklearn/covariance/_elliptic_envelope.py:145, in EllipticEnvelope.fit(self, X, y)
    134 def fit(self, X, y=None):
    135     """Fit the EllipticEnvelope model.
    136 
    137     Parameters
   (...)
    143         Not used, present for API consistency by convention.
    144     """
--> 145     super().fit(X)
    146     self.offset_ = np.percentile(-self.dist_, 100. * self.contamination)
    147     return self

File /opt/anaconda3/envs/benchmark/lib/python3.9/site-packages/sklearn/covariance/_robust_covariance.py:668, in MinCovDet.fit(self, X, y)
    666 self.dist_ = raw_dist
    667 # obtain consistency at normal models
--> 668 self.correct_covariance(X)
    669 # re-weight estimator
    670 self.reweight_covariance(X)

File /opt/anaconda3/envs/benchmark/lib/python3.9/site-packages/sklearn/covariance/_robust_covariance.py:705, in MinCovDet.correct_covariance(self, data)
    703 n_support = np.sum(self.support_)
    704 if n_support < n_samples and np.allclose(self.raw_covariance_, 0):
--> 705     raise ValueError('The covariance matrix of the support data '
    706                      'is equal to 0, try to increase support_fraction')
    707 correction = np.median(self.dist_) / chi2(data.shape[1]).isf(0.5)
    708 covariance_corrected = self.raw_covariance_ * correction

ValueError: The covariance matrix of the support data is equal to 0, try to increase support_fraction
```

### Versions

```shell
0.24.1
```

COMMENT:
@betatim - Would you need anything from my side? Just wanted to follow up and see if you have any interim solution for this.

Labels: Bug, Needs Info
Add tqdm integration for progress tracking in GridSearchCV

### Describe the workflow you want to enable

I believe it would be beneficial to integrate tqdm into GridSearchCV for more detailed and user-friendly progress tracking. While the verbose parameter provides some information, a progress bar could give users a better sense of how long the process will take. This could be particularly useful for large parameter grids that take a long time to process.

### Describe your proposed solution

One potential way to implement this would be to wrap the parameter grid in a tqdm object in the _run_search method. However, this would introduce a new dependency on tqdm, and it might not be consistent with the rest of the library's interface. I'm interested to hear the thoughts of the maintainers and other users on this idea.

### Describe alternatives you've considered, if relevant

_No response_

### Additional context

_No response_

COMMENT:
Hi @apollner, this feature has been requested many times. This is the kind of feature that will be possible with a callback api (https://github.com/scikit-learn/scikit-learn/pull/22000) which is work in progress.
I'm closing this issue as duplicate of https://github.com/scikit-learn/scikit-learn/issues/7574

Labels: New Feature, Needs Triage
Add tqdm integration for progress tracking in GridSearchCV

### Describe the workflow you want to enable

I believe it would be beneficial to integrate tqdm into GridSearchCV for more detailed and user-friendly progress tracking. While the verbose parameter provides some information, a progress bar could give users a better sense of how long the process will take. This could be particularly useful for large parameter grids that take a long time to process.

### Describe your proposed solution

One potential way to implement this would be to wrap the parameter grid in a tqdm object in the _run_search method. However, this would introduce a new dependency on tqdm, and it might not be consistent with the rest of the library's interface. I'm interested to hear the thoughts of the maintainers and other users on this idea.

### Describe alternatives you've considered, if relevant

_No response_

### Additional context

_No response_

COMMENT:
@jeremiedbb  will wait for the callback functionality to drop soon!

Forever wanted to use `tqdm` with `GridSearchCV`.

Labels: New Feature, Needs Triage
Add tqdm integration for progress tracking in GridSearchCV

### Describe the workflow you want to enable

I believe it would be beneficial to integrate tqdm into GridSearchCV for more detailed and user-friendly progress tracking. While the verbose parameter provides some information, a progress bar could give users a better sense of how long the process will take. This could be particularly useful for large parameter grids that take a long time to process.

### Describe your proposed solution

One potential way to implement this would be to wrap the parameter grid in a tqdm object in the _run_search method. However, this would introduce a new dependency on tqdm, and it might not be consistent with the rest of the library's interface. I'm interested to hear the thoughts of the maintainers and other users on this idea.

### Describe alternatives you've considered, if relevant

_No response_

### Additional context

_No response_

COMMENT:
Up for this request. It would be best to just pass tqdm object to GridSearchCV, or to request it with a flag, thank you.

Labels: New Feature, Needs Triage
GridSearchCV for neural networks that allows evaluation of the validation set after every epoch

### Describe the workflow you want to enable

I would like to be able to perform a grid search of hyperparemeters for a neural network using crossvalidation such as with GridSearchCV. I was able to set everything up in Scikit-Learn and the KerasRegressor wrapper but I then found out that my model was not evaluating on the validation set until the end of the number of epochs I set. This does not enable to track training and validation loss throughout training hence, assessing the performance of my model and taking action to e.g., use early stopping to prevent overfitting.

### Describe your proposed solution

enable GridSearchCV to pass the validation data to the estimator during fitting so validation loss can be tracked throughout training. Actually, If I understand how GridSearchCV works correctly, I think that perhaps an entire new DeepLearningGridSearchCV class would be more appropriate because I think that GridSearchCV follows (at least roughly) these steps:

1. Create data folds using CV strategy provided.
2.  fit training data to estimator (pipeline including scaling and model in my case)
3.  evaluate on validation data.

The process I was looking for would merge steps 2 and 3 so that:

1. Create data folds using CV strategy provided.
2.  fit training and validation data to estimator so validation data can be used for model assessment after every epoch throughout training

### Describe alternatives you've considered, if relevant

 In my case, I was training an autoencoder so both my X and y data needed to be scaled using the scaling factors calculated on the training data. I've created a custom scoring function to do so but my main problem seemed to be requiring GridSearchCV to pass the validation data during training  andhaving to pass more than one output from one step to another within the pipeline.

### Additional context

Has anyone done anything similar or is this something that would be possible?

Thanks

COMMENT:
I'm not sure if I understand this correctly. You want to pass a pre-defined validation set to the model or does the validation set need to be set and is different for each cv split? 

If the validation set is known in advance, you can already pass it as a fit param.

Labels: New Feature, Needs Triage
GridSearchCV for neural networks that allows evaluation of the validation set after every epoch

### Describe the workflow you want to enable

I would like to be able to perform a grid search of hyperparemeters for a neural network using crossvalidation such as with GridSearchCV. I was able to set everything up in Scikit-Learn and the KerasRegressor wrapper but I then found out that my model was not evaluating on the validation set until the end of the number of epochs I set. This does not enable to track training and validation loss throughout training hence, assessing the performance of my model and taking action to e.g., use early stopping to prevent overfitting.

### Describe your proposed solution

enable GridSearchCV to pass the validation data to the estimator during fitting so validation loss can be tracked throughout training. Actually, If I understand how GridSearchCV works correctly, I think that perhaps an entire new DeepLearningGridSearchCV class would be more appropriate because I think that GridSearchCV follows (at least roughly) these steps:

1. Create data folds using CV strategy provided.
2.  fit training data to estimator (pipeline including scaling and model in my case)
3.  evaluate on validation data.

The process I was looking for would merge steps 2 and 3 so that:

1. Create data folds using CV strategy provided.
2.  fit training and validation data to estimator so validation data can be used for model assessment after every epoch throughout training

### Describe alternatives you've considered, if relevant

 In my case, I was training an autoencoder so both my X and y data needed to be scaled using the scaling factors calculated on the training data. I've created a custom scoring function to do so but my main problem seemed to be requiring GridSearchCV to pass the validation data during training  andhaving to pass more than one output from one step to another within the pipeline.

### Additional context

Has anyone done anything similar or is this something that would be possible?

Thanks

COMMENT:
I'm not sure if what you're suggesting is a good idea, or if it's better to have a validation set, which is the same for all models.

Regardless, I don't think we'll develop this feature in `*SearchCV` the way you explain. And if you use a pre-defined validation set, you can already pass it to the models as a fit parameter.

So I'm going to close this issue as not planned for now (unless others feel differently).

Labels: New Feature, Needs Triage
⚠️ CI failed on Ubuntu_Jammy_Jellyfish.py38_conda_forge_openblas_ubuntu_2204 ⚠️

**CI is still failing on [Ubuntu_Jammy_Jellyfish.py38_conda_forge_openblas_ubuntu_2204](https://dev.azure.com/scikit-learn/scikit-learn/_build/results?buildId=55614&view=logs&j=7d13af21-9cb9-5d40-483b-ea0f074409c6)** (Jun 07, 2023)
- test_fastica_simple[42-float32-False]

COMMENT:
## CI is no longer failing! ✅

[Successful run](https://dev.azure.com/scikit-learn/scikit-learn/_build/results?buildId=55657&view=logs&j=7d13af21-9cb9-5d40-483b-ea0f074409c6) on Jun 08, 2023

Labels: Needs Triage
DOC add __sklearn_is_fitted__ and others to Developer guide

### Describe the issue linked to the documentation

It would be nice to add the internal developer "APIs" like `__sklearn_is_fitted__` to the user guide section [Developing scikit-learn estimators](https://scikit-learn.org/dev/developers/develop.html#developing-scikit-learn-estimators).

### Suggest a potential alternative/fix

_No response_

COMMENT:
Is there requirement to add docs for `check_is_fitted` from the [sklearn.utils.validation.check_is_fitted](https://scikit-learn.org/stable/modules/generated/sklearn.utils.validation.check_is_fitted.html) or docs for the `__sklearn_is_fitted__` method for custom estimators to check if an estimator has been fitted with data or not:

```
from sklearn.base import BaseEstimator

class CustomEstimator(BaseEstimator):
    def __init__(self):
        self.is_fitted_ = False
    
    def fit(self, X, y):
        # Perform fitting logic here
        self.is_fitted_ = True
        
    def predict(self, X):
        if not self.__sklearn_is_fitted__():
            raise ValueError("Estimator has not been fitted.")
        
        # Perform prediction logic here
    
    def __sklearn_is_fitted__(self):
        return self.is_fitted_

# Create an instance of the CustomEstimator
estimator = CustomEstimator()

# Check if the estimator is fitted
print(estimator.__sklearn_is_fitted__())  # Output: False

# Fit the estimator with data
estimator.fit(X, y)

# Check if the estimator is fitted again
print(estimator.__sklearn_is_fitted__())  # Output: True

# Use the estimator to make predictions
estimator.predict(X)
```

Labels: Documentation
DOC add __sklearn_is_fitted__ and others to Developer guide

### Describe the issue linked to the documentation

It would be nice to add the internal developer "APIs" like `__sklearn_is_fitted__` to the user guide section [Developing scikit-learn estimators](https://scikit-learn.org/dev/developers/develop.html#developing-scikit-learn-estimators).

### Suggest a potential alternative/fix

_No response_

COMMENT:
I currently don’t know, I only think it‘s not a requirement. That‘s why I‘d like to have it documented.

Labels: Documentation
DOC add __sklearn_is_fitted__ and others to Developer guide

### Describe the issue linked to the documentation

It would be nice to add the internal developer "APIs" like `__sklearn_is_fitted__` to the user guide section [Developing scikit-learn estimators](https://scikit-learn.org/dev/developers/develop.html#developing-scikit-learn-estimators).

### Suggest a potential alternative/fix

_No response_

COMMENT:
The documentation for `check_is_fitted` is very sparse on `__sklearn_is_fitted__`. I suggest:

- adding an example where we use `__sklearn_is_fitted__`
- link to the example from the `check_is_fitted` docstring
- mention `__sklearn_is_fitted__` in the developing custom estimator guide with a link to the example.

Labels: Documentation
DOC add __sklearn_is_fitted__ and others to Developer guide

### Describe the issue linked to the documentation

It would be nice to add the internal developer "APIs" like `__sklearn_is_fitted__` to the user guide section [Developing scikit-learn estimators](https://scikit-learn.org/dev/developers/develop.html#developing-scikit-learn-estimators).

### Suggest a potential alternative/fix

_No response_

COMMENT:
So, shall I go ahead with the enhancement to the `check_is_fitted` docstring of using `__sklearn_is_fitted__` and then I will mention that example through that link in the [Developing scikit-learn estimators](https://scikit-learn.org/dev/developers/develop.html#developing-scikit-learn-estimators).

Labels: Documentation
DOC add __sklearn_is_fitted__ and others to Developer guide

### Describe the issue linked to the documentation

It would be nice to add the internal developer "APIs" like `__sklearn_is_fitted__` to the user guide section [Developing scikit-learn estimators](https://scikit-learn.org/dev/developers/develop.html#developing-scikit-learn-estimators).

### Suggest a potential alternative/fix

_No response_

COMMENT:
Yes, sounds good. I'm just not sure if the example needs to be in the docstring or a separate file in `examples/miscelaneous` folder.

Labels: Documentation
DOC add __sklearn_is_fitted__ and others to Developer guide

### Describe the issue linked to the documentation

It would be nice to add the internal developer "APIs" like `__sklearn_is_fitted__` to the user guide section [Developing scikit-learn estimators](https://scikit-learn.org/dev/developers/develop.html#developing-scikit-learn-estimators).

### Suggest a potential alternative/fix

_No response_

COMMENT:
I got your point, I will put the example in `examples/miscelaneous` and mention that example on both of the docstrings. And after that we will look how to make `check_is_fitted` docstring more denser.

Labels: Documentation
DOC add __sklearn_is_fitted__ and others to Developer guide

### Describe the issue linked to the documentation

It would be nice to add the internal developer "APIs" like `__sklearn_is_fitted__` to the user guide section [Developing scikit-learn estimators](https://scikit-learn.org/dev/developers/develop.html#developing-scikit-learn-estimators).

### Suggest a potential alternative/fix

_No response_

COMMENT:
At this point, I think it's worth adding a new example section for "Developing Estimators".

Labels: Documentation
Improve error message with pandas output and sparse data

Currently, we raise an error:

```
ValueError: Pandas output does not support sparse data.
```

when a transformer does output sparse data and we try to wrap it inside a dataframe due to `set_output(transform="pandas")`. I assume that at this point, we could be extra nice and check if the transformer has a `sparse_output` keyword and advice changing it to `False`. We can also document that doing this change would be memory inefficient (it might blow up your RAM) but useful when it comes to data introspection.

What do you think about this proposal @thomasjpfan?

COMMENT:
I am +1 with adding a more informative error message for transformers with `sparse_output`.

Labels: New Feature
Improve error message with pandas output and sparse data

Currently, we raise an error:

```
ValueError: Pandas output does not support sparse data.
```

when a transformer does output sparse data and we try to wrap it inside a dataframe due to `set_output(transform="pandas")`. I assume that at this point, we could be extra nice and check if the transformer has a `sparse_output` keyword and advice changing it to `False`. We can also document that doing this change would be memory inefficient (it might blow up your RAM) but useful when it comes to data introspection.

What do you think about this proposal @thomasjpfan?

COMMENT:
Hi @thomasjpfan ,

I am new to open source and would love to work on this issue with your help and guidance.

Labels: New Feature
Improve error message with pandas output and sparse data

Currently, we raise an error:

```
ValueError: Pandas output does not support sparse data.
```

when a transformer does output sparse data and we try to wrap it inside a dataframe due to `set_output(transform="pandas")`. I assume that at this point, we could be extra nice and check if the transformer has a `sparse_output` keyword and advice changing it to `False`. We can also document that doing this change would be memory inefficient (it might blow up your RAM) but useful when it comes to data introspection.

What do you think about this proposal @thomasjpfan?

COMMENT:
@sqali You should start by going through the contribution guideline: https://scikit-learn.org/dev/developers/contributing.html

Regarding the code, you need to modify the error raised and change the unit tests that might break due to the change.

Labels: New Feature
⚠️ CI failed on linux_arm64_wheel ⚠️

**CI failed on [linux_arm64_wheel](https://cirrus-ci.com/build/4705054238703616)** (Jun 05, 2023)

COMMENT:
## CI is no longer failing! ✅

[Successful run](https://cirrus-ci.com/build/5999585902985216) on Jun 06, 2023

Labels: Needs Triage
⚠️ CI failed on linux_arm64_wheel ⚠️

**CI failed on [linux_arm64_wheel](https://cirrus-ci.com/build/4705054238703616)** (Jun 05, 2023)

COMMENT:
## CI is no longer failing! ✅

[Successful run](https://cirrus-ci.com/build/4705054238703616) on Jun 05, 2023

Labels: Needs Triage
⚠️ CI failed on linux_arm64_wheel ⚠️

**CI failed on [linux_arm64_wheel](https://cirrus-ci.com/build/5851815137247232)** (Jun 04, 2023)

COMMENT:
## CI is no longer failing! ✅

[Successful run](https://cirrus-ci.com/build/5851815137247232) on Jun 04, 2023

Labels: Needs Triage