Talip7/scikit-learn-issues-multilabel huggingface dataset:
DOC Example on model selection for Gaussian Mixture Models

### Describe the issue linked to the documentation

We have an example that illustrates how to use the BIC score to tune the number of components and the type of covariance matrix parametrization here:

https://scikit-learn.org/stable/auto_examples/mixture/plot_gmm_selection.html

However, the BIC score is not meant to be computed in a CV loop, but instead directly on the training set. So we should not use it with a `GridSearchCV` call. Indeed, the BIC score already penalizes the number of parameters depending on the number of data-points in the training set.

Instead, we should call the `GridSearchCV` on the default `.score` method of the GMM estimator, which computes the log-likelihood and is a perfectly fine metric to select the best model on held out data in a CV loop.

Note that we can keep computing the BIC score for all the hparam combinations but we should either do it in a single for loop (without train-test split), e.g.:

```python
from sklearn.model_selection import ParameterGrid
from sklearn.mixture import GaussianMixture
import pandas as pd
import numpy as np


n_samples = 500
rng = np.random.default_rng(0)
C = np.array([[0.0, -0.1], [1.7, 0.4]])
component_1 = rng.normal(size=(n_samples, 2)) @ C  # general
component_2 = 0.7 * rng.normal(size=(n_samples, 2)) + np.array([-4, 1])  # spherical
X = np.concatenate([component_1, component_2])

param_grid = {
    "n_components": np.arange(1, 7),
    "covariance_type": ["full", "tied", "diag", "spherical"],
}

bic_evaluations = []
for params in ParameterGrid(param_grid):
    bic_value = GaussianMixture(**params).fit(X).bic(X)
    bic_evaluations.append({**params, "BIC": bic_value})

bic_evaluations = pd.DataFrame(bic_evaluations).sort_values("BIC", ascending=True)
bic_evaluations.head()
```

So in summary I would recommend to:

- update the existing `GridSearchCV` code to use the `scoring=None` default that would use the built-in log-likelihood based model evaluation (averaged on the test sets of the CV loop);
- add a new section for BIC-based model selection with the for loop I proposed above as a computationally cheaper alternative to CV-based model selection with the log-likelihood.

We can then check that the two methods select the same model: 2 components with "full" (non-spherical) parametrization of the covariance matrix.

COMMENT:
I did paraphrase the same description you wrote the point was to support your recommendations only,I executed the code you mentioned. I am new to GitHub contributions, I am really sorry if I did something which us unwanted, will take care of it from next time.

Labels: Documentation
DOC Example on model selection for Gaussian Mixture Models

### Describe the issue linked to the documentation

We have an example that illustrates how to use the BIC score to tune the number of components and the type of covariance matrix parametrization here:

https://scikit-learn.org/stable/auto_examples/mixture/plot_gmm_selection.html

However, the BIC score is not meant to be computed in a CV loop, but instead directly on the training set. So we should not use it with a `GridSearchCV` call. Indeed, the BIC score already penalizes the number of parameters depending on the number of data-points in the training set.

Instead, we should call the `GridSearchCV` on the default `.score` method of the GMM estimator, which computes the log-likelihood and is a perfectly fine metric to select the best model on held out data in a CV loop.

Note that we can keep computing the BIC score for all the hparam combinations but we should either do it in a single for loop (without train-test split), e.g.:

```python
from sklearn.model_selection import ParameterGrid
from sklearn.mixture import GaussianMixture
import pandas as pd
import numpy as np


n_samples = 500
rng = np.random.default_rng(0)
C = np.array([[0.0, -0.1], [1.7, 0.4]])
component_1 = rng.normal(size=(n_samples, 2)) @ C  # general
component_2 = 0.7 * rng.normal(size=(n_samples, 2)) + np.array([-4, 1])  # spherical
X = np.concatenate([component_1, component_2])

param_grid = {
    "n_components": np.arange(1, 7),
    "covariance_type": ["full", "tied", "diag", "spherical"],
}

bic_evaluations = []
for params in ParameterGrid(param_grid):
    bic_value = GaussianMixture(**params).fit(X).bic(X)
    bic_evaluations.append({**params, "BIC": bic_value})

bic_evaluations = pd.DataFrame(bic_evaluations).sort_values("BIC", ascending=True)
bic_evaluations.head()
```

So in summary I would recommend to:

- update the existing `GridSearchCV` code to use the `scoring=None` default that would use the built-in log-likelihood based model evaluation (averaged on the test sets of the CV loop);
- add a new section for BIC-based model selection with the for loop I proposed above as a computationally cheaper alternative to CV-based model selection with the log-likelihood.

We can then check that the two methods select the same model: 2 components with "full" (non-spherical) parametrization of the covariance matrix.

COMMENT:
No problem. If you are new to contributing to scikit-learn, please make sure to follow the guidelines from: https://scikit-learn.org/dev/developers/index.html

And welcome to the project!

Labels: Documentation
DOC Example on model selection for Gaussian Mixture Models

### Describe the issue linked to the documentation

We have an example that illustrates how to use the BIC score to tune the number of components and the type of covariance matrix parametrization here:

https://scikit-learn.org/stable/auto_examples/mixture/plot_gmm_selection.html

However, the BIC score is not meant to be computed in a CV loop, but instead directly on the training set. So we should not use it with a `GridSearchCV` call. Indeed, the BIC score already penalizes the number of parameters depending on the number of data-points in the training set.

Instead, we should call the `GridSearchCV` on the default `.score` method of the GMM estimator, which computes the log-likelihood and is a perfectly fine metric to select the best model on held out data in a CV loop.

Note that we can keep computing the BIC score for all the hparam combinations but we should either do it in a single for loop (without train-test split), e.g.:

```python
from sklearn.model_selection import ParameterGrid
from sklearn.mixture import GaussianMixture
import pandas as pd
import numpy as np


n_samples = 500
rng = np.random.default_rng(0)
C = np.array([[0.0, -0.1], [1.7, 0.4]])
component_1 = rng.normal(size=(n_samples, 2)) @ C  # general
component_2 = 0.7 * rng.normal(size=(n_samples, 2)) + np.array([-4, 1])  # spherical
X = np.concatenate([component_1, component_2])

param_grid = {
    "n_components": np.arange(1, 7),
    "covariance_type": ["full", "tied", "diag", "spherical"],
}

bic_evaluations = []
for params in ParameterGrid(param_grid):
    bic_value = GaussianMixture(**params).fit(X).bic(X)
    bic_evaluations.append({**params, "BIC": bic_value})

bic_evaluations = pd.DataFrame(bic_evaluations).sort_values("BIC", ascending=True)
bic_evaluations.head()
```

So in summary I would recommend to:

- update the existing `GridSearchCV` code to use the `scoring=None` default that would use the built-in log-likelihood based model evaluation (averaged on the test sets of the CV loop);
- add a new section for BIC-based model selection with the for loop I proposed above as a computationally cheaper alternative to CV-based model selection with the log-likelihood.

We can then check that the two methods select the same model: 2 components with "full" (non-spherical) parametrization of the covariance matrix.

COMMENT:
I will go through the document thoroughly, thanks for the help.

Labels: Documentation
DOC Example on model selection for Gaussian Mixture Models

### Describe the issue linked to the documentation

We have an example that illustrates how to use the BIC score to tune the number of components and the type of covariance matrix parametrization here:

https://scikit-learn.org/stable/auto_examples/mixture/plot_gmm_selection.html

However, the BIC score is not meant to be computed in a CV loop, but instead directly on the training set. So we should not use it with a `GridSearchCV` call. Indeed, the BIC score already penalizes the number of parameters depending on the number of data-points in the training set.

Instead, we should call the `GridSearchCV` on the default `.score` method of the GMM estimator, which computes the log-likelihood and is a perfectly fine metric to select the best model on held out data in a CV loop.

Note that we can keep computing the BIC score for all the hparam combinations but we should either do it in a single for loop (without train-test split), e.g.:

```python
from sklearn.model_selection import ParameterGrid
from sklearn.mixture import GaussianMixture
import pandas as pd
import numpy as np


n_samples = 500
rng = np.random.default_rng(0)
C = np.array([[0.0, -0.1], [1.7, 0.4]])
component_1 = rng.normal(size=(n_samples, 2)) @ C  # general
component_2 = 0.7 * rng.normal(size=(n_samples, 2)) + np.array([-4, 1])  # spherical
X = np.concatenate([component_1, component_2])

param_grid = {
    "n_components": np.arange(1, 7),
    "covariance_type": ["full", "tied", "diag", "spherical"],
}

bic_evaluations = []
for params in ParameterGrid(param_grid):
    bic_value = GaussianMixture(**params).fit(X).bic(X)
    bic_evaluations.append({**params, "BIC": bic_value})

bic_evaluations = pd.DataFrame(bic_evaluations).sort_values("BIC", ascending=True)
bic_evaluations.head()
```

So in summary I would recommend to:

- update the existing `GridSearchCV` code to use the `scoring=None` default that would use the built-in log-likelihood based model evaluation (averaged on the test sets of the CV loop);
- add a new section for BIC-based model selection with the for loop I proposed above as a computationally cheaper alternative to CV-based model selection with the log-likelihood.

We can then check that the two methods select the same model: 2 components with "full" (non-spherical) parametrization of the covariance matrix.

COMMENT:
@Uvi-12 this plan looks good. It's quite in line with the suggestion I made in the previous review: https://github.com/scikit-learn/scikit-learn/pull/30329#discussion_r1856662564

> ensuring proper model selection.

"Proper" is a bit strong. BIC is still some kind of heuristic model selection tool to me. I would rather say, "leveraging BIC's ability to trade data fidelity for model complexity".

Labels: Documentation
DOC Example on model selection for Gaussian Mixture Models

### Describe the issue linked to the documentation

We have an example that illustrates how to use the BIC score to tune the number of components and the type of covariance matrix parametrization here:

https://scikit-learn.org/stable/auto_examples/mixture/plot_gmm_selection.html

However, the BIC score is not meant to be computed in a CV loop, but instead directly on the training set. So we should not use it with a `GridSearchCV` call. Indeed, the BIC score already penalizes the number of parameters depending on the number of data-points in the training set.

Instead, we should call the `GridSearchCV` on the default `.score` method of the GMM estimator, which computes the log-likelihood and is a perfectly fine metric to select the best model on held out data in a CV loop.

Note that we can keep computing the BIC score for all the hparam combinations but we should either do it in a single for loop (without train-test split), e.g.:

```python
from sklearn.model_selection import ParameterGrid
from sklearn.mixture import GaussianMixture
import pandas as pd
import numpy as np


n_samples = 500
rng = np.random.default_rng(0)
C = np.array([[0.0, -0.1], [1.7, 0.4]])
component_1 = rng.normal(size=(n_samples, 2)) @ C  # general
component_2 = 0.7 * rng.normal(size=(n_samples, 2)) + np.array([-4, 1])  # spherical
X = np.concatenate([component_1, component_2])

param_grid = {
    "n_components": np.arange(1, 7),
    "covariance_type": ["full", "tied", "diag", "spherical"],
}

bic_evaluations = []
for params in ParameterGrid(param_grid):
    bic_value = GaussianMixture(**params).fit(X).bic(X)
    bic_evaluations.append({**params, "BIC": bic_value})

bic_evaluations = pd.DataFrame(bic_evaluations).sort_values("BIC", ascending=True)
bic_evaluations.head()
```

So in summary I would recommend to:

- update the existing `GridSearchCV` code to use the `scoring=None` default that would use the built-in log-likelihood based model evaluation (averaged on the test sets of the CV loop);
- add a new section for BIC-based model selection with the for loop I proposed above as a computationally cheaper alternative to CV-based model selection with the log-likelihood.

We can then check that the two methods select the same model: 2 components with "full" (non-spherical) parametrization of the covariance matrix.

COMMENT:
> @Uvi-12 this plan looks good. It's quite in line with the suggestion I made in the previous review: [#30329 (comment)](https://github.com/scikit-learn/scikit-learn/pull/30329#discussion_r1856662564)
> 
> > ensuring proper model selection.
> 
> "Proper" is a bit strong. BIC is still some kind of heuristic model selection tool to me. I would rather say, "leveraging BIC's ability to trade data fidelity for model complexity".

Thank you for the feedback, I will update it as you said. I am relatively new to the code base can you please help me with the code to be added to the documentation?

Labels: Documentation
Error in impute/_base.py _most_frequent when array contains None

### Describe the bug

TypeError: '<' not supported between instances of 'NoneType' and 'str'
when calculating min in impute/_base.py

most_frequent_value = min(
                value
                for value, count in counter.items()
                if count == most_frequent_count
            )

when array has None value as the most frequent one.

### Steps/Code to Reproduce

array = numpy.array(['a','b',None,None])

### Expected Results

most frequent: 'a'
imputed array: ['a','b','a','a']

### Actual Results

TypeError: '<' not supported between instances of 'NoneType' and 'str'

COMMENT:
fixed by changing two lines:

    #if array.size > 0:
    if np.array([a for a in array if a is not None]).size > 0:

           #counter = Counter(array)
            counter = Counter([a for a in array if a is not None])


### Versions

```shell
System:
    python: 3.12.3 (main, Nov  6 2024, 18:32:19) [GCC 13.2.0]
executable: /home/padiadev/venv/bin/python
   machine: Linux-6.8.0-45-generic-x86_64-with-glibc2.39

Python dependencies:
      sklearn: 1.5.2
          pip: 24.0
   setuptools: None
        numpy: 1.26.4
        scipy: 1.14.1
       Cython: None
       pandas: 2.2.3
   matplotlib: 3.9.2
       joblib: 1.4.2
threadpoolctl: 3.5.0

Built with OpenMP: True

threadpoolctl info:
       user_api: blas
   internal_api: openblas
    num_threads: 2
         prefix: libopenblas
       filepath: /home/padiadev/venv/lib/python3.12/site-packages/numpy.libs/libopenblas64_p-r0-0cf96a72.3.23.dev.so
        version: 0.3.23.dev
threading_layer: pthreads
   architecture: Haswell

       user_api: blas
   internal_api: openblas
    num_threads: 2
         prefix: libscipy_openblas
       filepath: /home/padiadev/venv/lib/python3.12/site-packages/scipy.libs/libscipy_openblas-c128ec02.so
        version: 0.3.27.dev
threading_layer: pthreads
   architecture: Haswell

       user_api: openmp
   internal_api: openmp
    num_threads: 2
         prefix: libgomp
       filepath: /home/padiadev/venv/lib/python3.12/site-packages/scikit_learn.libs/libgomp-a34b3233.so.1.0.0
        version: None
```

COMMENT:
I cannot reproduce. Did you set properly the `missing_values` parameters:

```python
import numpy as np
from sklearn.impute import SimpleImputer

array = np.array(["a", "b", None, None])
array = array.reshape(-1, 1)
imputer = SimpleImputer(missing_values=None, strategy="most_frequent")
imputer.fit_transform(array)
```

```
array([['a'],
       ['b'],
       ['a'],
       ['a']], dtype=object)
```

Labels: Bug, Needs Triage
⚠️ CI failed on Linux_Nightly.pylatest_pip_scipy_dev (last failure: Dec 05, 2024) ⚠️

**CI is still failing on [Linux_Nightly.pylatest_pip_scipy_dev](https://dev.azure.com/scikit-learn/scikit-learn/_build/results?buildId=72598&view=logs&j=dfe99b15-50db-5d7b-b1e9-4105c42527cf)** (Dec 05, 2024)
- test_partial_dependence_binary_model_grid_resolution[features0-10-10]

COMMENT:
Apparently, we try to affect a float value generated from the grid into a column of integer and it will fail in Pandas 3.0. We could change the test such that we don't trigger the issue.

However, we should revisit this use case: potentially a user pass integral values to represent categories; so sampling floating values do not make sense. So we might want to always raise an error mentioning that you should either convert in to float or declare it as a categorical feature.

I have to look at it in more details.

Labels: Bug
⚠️ CI failed on Linux_Nightly.pylatest_pip_scipy_dev (last failure: Dec 05, 2024) ⚠️

**CI is still failing on [Linux_Nightly.pylatest_pip_scipy_dev](https://dev.azure.com/scikit-learn/scikit-learn/_build/results?buildId=72598&view=logs&j=dfe99b15-50db-5d7b-b1e9-4105c42527cf)** (Dec 05, 2024)
- test_partial_dependence_binary_model_grid_resolution[features0-10-10]

COMMENT:
## CI is no longer failing! ✅

[Successful run](https://dev.azure.com/scikit-learn/scikit-learn/_build/results?buildId=72695&view=logs&j=dfe99b15-50db-5d7b-b1e9-4105c42527cf) on Dec 09, 2024

Labels: Bug
⚠️ CI failed on Linux_Nightly.pylatest_pip_scipy_dev (last failure: Dec 05, 2024) ⚠️

**CI is still failing on [Linux_Nightly.pylatest_pip_scipy_dev](https://dev.azure.com/scikit-learn/scikit-learn/_build/results?buildId=72598&view=logs&j=dfe99b15-50db-5d7b-b1e9-4105c42527cf)** (Dec 05, 2024)
- test_partial_dependence_binary_model_grid_resolution[features0-10-10]

COMMENT:
This seems weird that the failure does not seem deterministic ...

- it failed on November 21 (first observed failure), November 22, November 23, November 28
- it passed on the other days

Labels: Bug
⚠️ CI failed on Linux_Nightly.pylatest_pip_scipy_dev (last failure: Dec 05, 2024) ⚠️

**CI is still failing on [Linux_Nightly.pylatest_pip_scipy_dev](https://dev.azure.com/scikit-learn/scikit-learn/_build/results?buildId=72598&view=logs&j=dfe99b15-50db-5d7b-b1e9-4105c42527cf)** (Dec 05, 2024)
- test_partial_dependence_binary_model_grid_resolution[features0-10-10]

COMMENT:
This is probably a consequence of https://github.com/scipy/scipy/pull/21824. I need to read more carefully, but I think this is not a bug, and we have to adapt our test.

Labels: Bug
⚠️ CI failed on Linux_Nightly.pylatest_pip_scipy_dev (last failure: Dec 05, 2024) ⚠️

**CI is still failing on [Linux_Nightly.pylatest_pip_scipy_dev](https://dev.azure.com/scikit-learn/scikit-learn/_build/results?buildId=72598&view=logs&j=dfe99b15-50db-5d7b-b1e9-4105c42527cf)** (Dec 05, 2024)
- test_partial_dependence_binary_model_grid_resolution[features0-10-10]

COMMENT:
I can reproduce the `test_partial_dependence_binary_model_grid_resolution[features0-10-10]` but it is not deterministic. With `pytest-repeat` running the test a 100 times fails roughly 10 times.

```
 pytest sklearn/inspection/tests/test_partial_dependence.py -k 'test_partial_dependence_binary_model_grid_resolution and 10-10' --count 100
```

```
FAILED sklearn/inspection/tests/test_partial_dependence.py::test_partial_dependence_binary_model_grid_resolution[features0-10-10-16-100] - TypeError: Invalid value '0.41000000000000014' for dtype 'int64'
FAILED sklearn/inspection/tests/test_partial_dependence.py::test_partial_dependence_binary_model_grid_resolution[features0-10-10-18-100] - TypeError: Invalid value '0.41000000000000014' for dtype 'int64'
FAILED sklearn/inspection/tests/test_partial_dependence.py::test_partial_dependence_binary_model_grid_resolution[features0-10-10-30-100] - TypeError: Invalid value '0.9544444444444449' for dtype 'int64'
FAILED sklearn/inspection/tests/test_partial_dependence.py::test_partial_dependence_binary_model_grid_resolution[features0-10-10-37-100] - TypeError: Invalid value '0.41000000000000014' for dtype 'int64'
FAILED sklearn/inspection/tests/test_partial_dependence.py::test_partial_dependence_binary_model_grid_resolution[features0-10-102-47-100] - TypeError: Invalid value '0.41000000000000014' for dtype 'int64'
FAILED sklearn/inspection/tests/test_partial_dependence.py::test_partial_dependence_binary_model_grid_resolution[features0-10-10-52-100] - TypeError: Invalid value '0.41000000000000014' for dtype 'int64'
FAILED sklearn/inspection/tests/test_partial_dependence.py::test_partial_dependence_binary_model_grid_resolution[features0-10-10-67-100] - TypeError: Invalid value '0.9544444444444449' for dtype 'int64'
FAILED sklearn/inspection/tests/test_partial_dependence.py::test_partial_dependence_binary_model_grid_resolution[features0-10-10-74-100] - TypeError: Invalid value '0.8888888888888888' for dtype 'int64'
FAILED sklearn/inspection/tests/test_partial_dependence.py::test_partial_dependence_binary_model_grid_resolution[features0-10-10-84-100] - TypeError: Invalid value '0.9544444444444449' for dtype 'int64'
FAILED sklearn/inspection/tests/test_partial_dependence.py::test_partial_dependence_binary_model_grid_resolution[features0-10-10-85-100] - TypeError: Invalid value '0.41000000000000014' for dtype 'int64'
```

Labels: Bug
⚠️ CI failed on Linux_Nightly.pylatest_pip_scipy_dev (last failure: Dec 05, 2024) ⚠️

**CI is still failing on [Linux_Nightly.pylatest_pip_scipy_dev](https://dev.azure.com/scikit-learn/scikit-learn/_build/results?buildId=72598&view=logs&j=dfe99b15-50db-5d7b-b1e9-4105c42527cf)** (Dec 05, 2024)
- test_partial_dependence_binary_model_grid_resolution[features0-10-10]

COMMENT:
@lesteve I have found a fix for `test_partial_dependence_binary_model_grid_resolution`. It's a real bug. I will open a PR.

Labels: Bug
⚠️ CI failed on Linux_Nightly.pylatest_pip_scipy_dev (last failure: Dec 05, 2024) ⚠️

**CI is still failing on [Linux_Nightly.pylatest_pip_scipy_dev](https://dev.azure.com/scikit-learn/scikit-learn/_build/results?buildId=72598&view=logs&j=dfe99b15-50db-5d7b-b1e9-4105c42527cf)** (Dec 05, 2024)
- test_partial_dependence_binary_model_grid_resolution[features0-10-10]

COMMENT:
This has been fixed by #30409.

Labels: Bug
'Section Navigation' bar missing from stable documentation website on several pages

### Describe the issue linked to the documentation

When on the stable version of the documentation website the 'Section Navigation' header on the left side of the page remains present, but the navigation bar contents disappear. While on the dev page the feature functions as expected.
It should be noted this issue is inconsistent. Some stable pages list the section navigation and work perfectly fine ([like this one here](https://scikit-learn.org/stable/modules/tree.html)) while others do not.

This presents an issue as some links take the user to the stable version and others the dev version.

For example: [Present Here](https://scikit-learn.org/dev/developers/contributing.html#submitting-a-bug-report-or-a-feature-request), [Absent Here](https://scikit-learn.org/stable/developers/contributing.html#submitting-a-bug-report-or-a-feature-request)

<img width="2046" alt="Screenshot 2024-11-19 at 19 22 09" src="https://github.com/user-attachments/assets/fe9e7e2a-4c04-4245-92e9-08dd697882ec">
<img width="2048" alt="Screenshot 2024-11-19 at 19 22 39" src="https://github.com/user-attachments/assets/378dc707-0dde-4abf-bc4a-4f38a0ff9513">

Discovered running on Chrome Browser Version 130.0.6723.117

COMMENT:
Basically this is a bug from `pydata-sphinx-theme`: https://github.com/pydata/pydata-sphinx-theme/issues/1662

Here, we could temporary do a hack where we disable the navigation manually until it is the issue get fix.

Labels: Documentation, Moderate