Talip7/scikit-learn-issues-multilabel huggingface dataset:
<!DOCTYPE HTML PUBLIC - 503 Server unavailable error - ALOPS Release Deployment failed

One of our customer's PROD environment release pipeline failed its first release. 

Error: 
##[error]Exception in BCConnector.GetAPIData: <!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01//EN""[http://www.w3.org/TR/html4/strict.dtd](https://eur02.safelinks.protection.outlook.com/?url=http%3A%2F%2Fwww.w3.org%2FTR%2Fhtml4%2Fstrict.dtd&data=05%7C01%7Cjkarri%40mprise.nl%7Caa0c2eecb7ac4a46a06808dad9ba11ca%7Cbebc3a1cdc604cd89669a36303f40ff4%7C0%7C0%7C638061690438858501%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=bl0fP4H5O9gJHqgcR%2Bma%2B1U4G%2BsjvjeTvDJd1UanoE4%3D&reserved=0)">
##[error]Cannot find an overload for "GetExtensions" and the argument count: "1".
<img width="518" alt="Release fail error" src="https://user-images.githubusercontent.com/61264260/210202840-fb70045c-bc29-4700-85e4-429d5ec1585c.png">

Question: I'm first time facing this DOCTYPE HTML error. May I know what causes the "<!DOCTYPE HTML Public" error come? 
Or is this just server down or something?

What I checked from my end: 
1.	The servicetier for deployment/pipeline is running
2.	The DeployUser user does exist in the BC installation
3.	The DeployUser user does have the right License Type (in BC) - External User type
4.	The Microsoft API app _Exclude_APIV1_ is installed 
5.	The API endpoint port has been added in the firewall of the environment where the BC installation resides 
6.	The ALOps External Deployer has been installed
7.	The user account (DeployUser) credentials on the pipeline are right
8.	The NST SSL certificate is installed 
9.	On the NST the API services have been enabled
10. Recreated by cloning the TEST release pipeline and the result is same. 

More Information: 
We got our TEST environment release pipeline deploying successfully. 
We use O365 logins for web client.

COMMENT:
Hi Adrinjalali, this issue should be posted in HodorNV/ALOps., and I did. Thank you. 
https://github.com/HodorNV/ALOps/issues/598

Labels: Needs Triage
Matthews correlation coefficient gives wrong results for single-class result

### Describe the bug

for a single class input, (ie perfect correlation) MCC returns 0, when it should return 1.

since MCC is a special case of pearson correlation, it should fall back to pearson when appropriate

### Steps/Code to Reproduce

from sklearn.metrics import matthews_corrcoef
y_true = [1, 1, 1, 1]
y_pred = [1, 1, 1, 1] 
matthews_corrcoef(y_true, y_pred)
Out  [9]: 0.0

### Expected Results

output should be 1

### Actual Results

output is 0

### Versions

```shell
import sklearn; sklearn.show_versions()

System:
    python: 3.9.12 (main, Apr  5 2022, 01:53:17)  [Clang 12.0.0 ]
   machine: macOS-10.16-x86_64-i386-64bit

Python dependencies:
          pip: 22.3.1
   setuptools: 65.5.1
      sklearn: 1.0.2
        numpy: 1.22.3
        scipy: 1.9.3
       Cython: 0.29.32
       pandas: 1.5.2
   matplotlib: 3.6.2
       joblib: 1.2.0
threadpoolctl: 3.1.0

Built with OpenMP: True
/Users/phild1/opt/anaconda3/lib/python3.9/site-packages/_distutils_hack/__init__.py:33: UserWarning: Setuptools is replacing distutils.
  warnings.warn("Setuptools is replacing distutils.")
```

COMMENT:
I think that we are in a corner case. We should probably add a parameter `zero_division` to control the output value and provide a warning in the default case. Regarding if we should output 0.0 or 1.0, we should be looking at the other metric to have something consistent.

Labels: Bug, module:metrics
Matthews correlation coefficient gives wrong results for single-class result

### Describe the bug

for a single class input, (ie perfect correlation) MCC returns 0, when it should return 1.

since MCC is a special case of pearson correlation, it should fall back to pearson when appropriate

### Steps/Code to Reproduce

from sklearn.metrics import matthews_corrcoef
y_true = [1, 1, 1, 1]
y_pred = [1, 1, 1, 1] 
matthews_corrcoef(y_true, y_pred)
Out  [9]: 0.0

### Expected Results

output should be 1

### Actual Results

output is 0

### Versions

```shell
import sklearn; sklearn.show_versions()

System:
    python: 3.9.12 (main, Apr  5 2022, 01:53:17)  [Clang 12.0.0 ]
   machine: macOS-10.16-x86_64-i386-64bit

Python dependencies:
          pip: 22.3.1
   setuptools: 65.5.1
      sklearn: 1.0.2
        numpy: 1.22.3
        scipy: 1.9.3
       Cython: 0.29.32
       pandas: 1.5.2
   matplotlib: 3.6.2
       joblib: 1.2.0
threadpoolctl: 3.1.0

Built with OpenMP: True
/Users/phild1/opt/anaconda3/lib/python3.9/site-packages/_distutils_hack/__init__.py:33: UserWarning: Setuptools is replacing distutils.
  warnings.warn("Setuptools is replacing distutils.")
```

COMMENT:
@dsphilli1 I added support for this in my related PR: https://github.com/scikit-learn/scikit-learn/pull/23183, as per Guillaume's suggestion

Labels: Bug, module:metrics
Matthews correlation coefficient gives wrong results for single-class result

### Describe the bug

for a single class input, (ie perfect correlation) MCC returns 0, when it should return 1.

since MCC is a special case of pearson correlation, it should fall back to pearson when appropriate

### Steps/Code to Reproduce

from sklearn.metrics import matthews_corrcoef
y_true = [1, 1, 1, 1]
y_pred = [1, 1, 1, 1] 
matthews_corrcoef(y_true, y_pred)
Out  [9]: 0.0

### Expected Results

output should be 1

### Actual Results

output is 0

### Versions

```shell
import sklearn; sklearn.show_versions()

System:
    python: 3.9.12 (main, Apr  5 2022, 01:53:17)  [Clang 12.0.0 ]
   machine: macOS-10.16-x86_64-i386-64bit

Python dependencies:
          pip: 22.3.1
   setuptools: 65.5.1
      sklearn: 1.0.2
        numpy: 1.22.3
        scipy: 1.9.3
       Cython: 0.29.32
       pandas: 1.5.2
   matplotlib: 3.6.2
       joblib: 1.2.0
threadpoolctl: 3.1.0

Built with OpenMP: True
/Users/phild1/opt/anaconda3/lib/python3.9/site-packages/_distutils_hack/__init__.py:33: UserWarning: Setuptools is replacing distutils.
  warnings.warn("Setuptools is replacing distutils.")
```

COMMENT:
The decision in https://github.com/scikit-learn/scikit-learn/pull/19977 was based on a Wikipedia article that stated:
> If any of the four sums in the denominator is zero, the denominator can be arbitrarily set to one; this results in a Matthews correlation coefficient of zero, which can be shown to be the correct limiting value

While this is correct (and yields favorable metric behavior) when **only one** of the sums is zero, in the case two of them is zero (as in the case in this issue), the (multivariable calculus) limit does not exist and the MCC is 0 for perfect prediction. I've corrected the Wikipedia article accordingly. This might be especially problematic in case someone calculates the metric in batches and aggregates (not the best practice here, but people might do it), then this can change the results even is the data is not really a single class.

I'd suggest returning zero in the cases the limit is zero and a `nan` when it doesn't exist.

Labels: Bug, module:metrics
Matthews correlation coefficient gives wrong results for single-class result

### Describe the bug

for a single class input, (ie perfect correlation) MCC returns 0, when it should return 1.

since MCC is a special case of pearson correlation, it should fall back to pearson when appropriate

### Steps/Code to Reproduce

from sklearn.metrics import matthews_corrcoef
y_true = [1, 1, 1, 1]
y_pred = [1, 1, 1, 1] 
matthews_corrcoef(y_true, y_pred)
Out  [9]: 0.0

### Expected Results

output should be 1

### Actual Results

output is 0

### Versions

```shell
import sklearn; sklearn.show_versions()

System:
    python: 3.9.12 (main, Apr  5 2022, 01:53:17)  [Clang 12.0.0 ]
   machine: macOS-10.16-x86_64-i386-64bit

Python dependencies:
          pip: 22.3.1
   setuptools: 65.5.1
      sklearn: 1.0.2
        numpy: 1.22.3
        scipy: 1.9.3
       Cython: 0.29.32
       pandas: 1.5.2
   matplotlib: 3.6.2
       joblib: 1.2.0
threadpoolctl: 3.1.0

Built with OpenMP: True
/Users/phild1/opt/anaconda3/lib/python3.9/site-packages/_distutils_hack/__init__.py:33: UserWarning: Setuptools is replacing distutils.
  warnings.warn("Setuptools is replacing distutils.")
```

COMMENT:
Feel free to contribute :)

Labels: Bug, module:metrics
Matthews correlation coefficient gives wrong results for single-class result

### Describe the bug

for a single class input, (ie perfect correlation) MCC returns 0, when it should return 1.

since MCC is a special case of pearson correlation, it should fall back to pearson when appropriate

### Steps/Code to Reproduce

from sklearn.metrics import matthews_corrcoef
y_true = [1, 1, 1, 1]
y_pred = [1, 1, 1, 1] 
matthews_corrcoef(y_true, y_pred)
Out  [9]: 0.0

### Expected Results

output should be 1

### Actual Results

output is 0

### Versions

```shell
import sklearn; sklearn.show_versions()

System:
    python: 3.9.12 (main, Apr  5 2022, 01:53:17)  [Clang 12.0.0 ]
   machine: macOS-10.16-x86_64-i386-64bit

Python dependencies:
          pip: 22.3.1
   setuptools: 65.5.1
      sklearn: 1.0.2
        numpy: 1.22.3
        scipy: 1.9.3
       Cython: 0.29.32
       pandas: 1.5.2
   matplotlib: 3.6.2
       joblib: 1.2.0
threadpoolctl: 3.1.0

Built with OpenMP: True
/Users/phild1/opt/anaconda3/lib/python3.9/site-packages/_distutils_hack/__init__.py:33: UserWarning: Setuptools is replacing distutils.
  warnings.warn("Setuptools is replacing distutils.")
```

COMMENT:
The same way we introduced the `zero_division` in some metric, we can add the option and being consistent with other metrics.

Labels: Bug, module:metrics
Matthews correlation coefficient gives wrong results for single-class result

### Describe the bug

for a single class input, (ie perfect correlation) MCC returns 0, when it should return 1.

since MCC is a special case of pearson correlation, it should fall back to pearson when appropriate

### Steps/Code to Reproduce

from sklearn.metrics import matthews_corrcoef
y_true = [1, 1, 1, 1]
y_pred = [1, 1, 1, 1] 
matthews_corrcoef(y_true, y_pred)
Out  [9]: 0.0

### Expected Results

output should be 1

### Actual Results

output is 0

### Versions

```shell
import sklearn; sklearn.show_versions()

System:
    python: 3.9.12 (main, Apr  5 2022, 01:53:17)  [Clang 12.0.0 ]
   machine: macOS-10.16-x86_64-i386-64bit

Python dependencies:
          pip: 22.3.1
   setuptools: 65.5.1
      sklearn: 1.0.2
        numpy: 1.22.3
        scipy: 1.9.3
       Cython: 0.29.32
       pandas: 1.5.2
   matplotlib: 3.6.2
       joblib: 1.2.0
threadpoolctl: 3.1.0

Built with OpenMP: True
/Users/phild1/opt/anaconda3/lib/python3.9/site-packages/_distutils_hack/__init__.py:33: UserWarning: Setuptools is replacing distutils.
  warnings.warn("Setuptools is replacing distutils.")
```

COMMENT:
I think that we did step one of the bullet list given in: https://github.com/scikit-learn/scikit-learn/pull/23183#issuecomment-1411118385

A way forward is to propose small individual PR that makes it easier to review and be merged.

Labels: Bug, module:metrics
Matthews correlation coefficient gives wrong results for single-class result

### Describe the bug

for a single class input, (ie perfect correlation) MCC returns 0, when it should return 1.

since MCC is a special case of pearson correlation, it should fall back to pearson when appropriate

### Steps/Code to Reproduce

from sklearn.metrics import matthews_corrcoef
y_true = [1, 1, 1, 1]
y_pred = [1, 1, 1, 1] 
matthews_corrcoef(y_true, y_pred)
Out  [9]: 0.0

### Expected Results

output should be 1

### Actual Results

output is 0

### Versions

```shell
import sklearn; sklearn.show_versions()

System:
    python: 3.9.12 (main, Apr  5 2022, 01:53:17)  [Clang 12.0.0 ]
   machine: macOS-10.16-x86_64-i386-64bit

Python dependencies:
          pip: 22.3.1
   setuptools: 65.5.1
      sklearn: 1.0.2
        numpy: 1.22.3
        scipy: 1.9.3
       Cython: 0.29.32
       pandas: 1.5.2
   matplotlib: 3.6.2
       joblib: 1.2.0
threadpoolctl: 3.1.0

Built with OpenMP: True
/Users/phild1/opt/anaconda3/lib/python3.9/site-packages/_distutils_hack/__init__.py:33: UserWarning: Setuptools is replacing distutils.
  warnings.warn("Setuptools is replacing distutils.")
```

COMMENT:
> Feel free to contribute :)

Sure, I'll open a little PR over the weekend

Labels: Bug, module:metrics
ledoit_wolf_shrinkage is not documented but still publicly available

### Describe the issue linked to the documentation

`ledoit_wolf_shrinkage` is not documented, but publicly available.

See discussion in #24870 (https://github.com/scikit-learn/scikit-learn/pull/24870#discussion_r1019598803)

### Suggest a potential alternative/fix

It should be either documented or made private with deprecation

COMMENT:
I was looking at a PR linked to `oas` and we probably should not deprecate this function but instead document it. It could become handy.

Labels: Documentation
Allow for passing additional parameters to the estimator's `fit` method in `SequentialFeatureSelector`

### Describe the workflow you want to enable

I would like to be able to pass sample weights to the `fit` method of the estimator in `SequentialFeatureSelector`. (`SelectFromModel` has this feature as well.)

Looking at the code, it seems to me that [`sklearn.model_selection._validation._fit_and_score`](https://github.com/scikit-learn/scikit-learn/blob/2e481f114169396660f0051eee1bcf6bcddfd556/sklearn/model_selection/_validation.py#L662), which is where the actual scoring takes place, already supports sample weights in cross-validation (i.e., takes care of passing only the weights for samples that are actually part of the fold).

### Describe your proposed solution

Based on my current understanding, `SequentialFeatureSelector.fit()` would need to get an additional argument `fit_params`  (cf. [`SelectFromModel.fit()`](https://scikit-learn.org/stable/modules/generated/sklearn.feature_selection.SelectFromModel.html#sklearn.feature_selection.SelectFromModel.fit)) that is passed on to the `cross_val_score` function called in `SequentialFeatureSelector._get_best_new_feature_score`. Everything else appears to be in place already.

if that's all it takes, I'm happy to prepare a PR. But I might very well be overlooking issues or unintended consequences.

### Describe alternatives you've considered, if relevant

_No response_

### Additional context

_No response_

COMMENT:
We are most certainly interested to be able to pass additional parameters. The SLEP006 is intending to do so.

@adrinjalali do you think that we should be implemented this support now or we should await the implementation of the SLEP006 to land in `main` with proper support?

Labels: New Feature, module:feature_selection
Allow for passing additional parameters to the estimator's `fit` method in `SequentialFeatureSelector`

### Describe the workflow you want to enable

I would like to be able to pass sample weights to the `fit` method of the estimator in `SequentialFeatureSelector`. (`SelectFromModel` has this feature as well.)

Looking at the code, it seems to me that [`sklearn.model_selection._validation._fit_and_score`](https://github.com/scikit-learn/scikit-learn/blob/2e481f114169396660f0051eee1bcf6bcddfd556/sklearn/model_selection/_validation.py#L662), which is where the actual scoring takes place, already supports sample weights in cross-validation (i.e., takes care of passing only the weights for samples that are actually part of the fold).

### Describe your proposed solution

Based on my current understanding, `SequentialFeatureSelector.fit()` would need to get an additional argument `fit_params`  (cf. [`SelectFromModel.fit()`](https://scikit-learn.org/stable/modules/generated/sklearn.feature_selection.SelectFromModel.html#sklearn.feature_selection.SelectFromModel.fit)) that is passed on to the `cross_val_score` function called in `SequentialFeatureSelector._get_best_new_feature_score`. Everything else appears to be in place already.

if that's all it takes, I'm happy to prepare a PR. But I might very well be overlooking issues or unintended consequences.

### Describe alternatives you've considered, if relevant

_No response_

### Additional context

_No response_

COMMENT:
I had a look at SLEP006 and I believe its implementation is required, since passing `sample_weight` to the estimator's `fit()` method is possible but it is currently not possible to pass `sample_weight` to the `score()` method (cf. [`sklearn.model_selection._validation._fit_and_score`](https://github.com/scikit-learn/scikit-learn/blob/2e481f114169396660f0051eee1bcf6bcddfd556/sklearn/model_selection/_validation.py#L708)). That's something I hadn't considered yet.

Labels: New Feature, module:feature_selection
Allow for passing additional parameters to the estimator's `fit` method in `SequentialFeatureSelector`

### Describe the workflow you want to enable

I would like to be able to pass sample weights to the `fit` method of the estimator in `SequentialFeatureSelector`. (`SelectFromModel` has this feature as well.)

Looking at the code, it seems to me that [`sklearn.model_selection._validation._fit_and_score`](https://github.com/scikit-learn/scikit-learn/blob/2e481f114169396660f0051eee1bcf6bcddfd556/sklearn/model_selection/_validation.py#L662), which is where the actual scoring takes place, already supports sample weights in cross-validation (i.e., takes care of passing only the weights for samples that are actually part of the fold).

### Describe your proposed solution

Based on my current understanding, `SequentialFeatureSelector.fit()` would need to get an additional argument `fit_params`  (cf. [`SelectFromModel.fit()`](https://scikit-learn.org/stable/modules/generated/sklearn.feature_selection.SelectFromModel.html#sklearn.feature_selection.SelectFromModel.fit)) that is passed on to the `cross_val_score` function called in `SequentialFeatureSelector._get_best_new_feature_score`. Everything else appears to be in place already.

if that's all it takes, I'm happy to prepare a PR. But I might very well be overlooking issues or unintended consequences.

### Describe alternatives you've considered, if relevant

_No response_

### Additional context

_No response_

COMMENT:
Since there's only one sub-estimator here, and no routing like in the pipeline is required, we can go ahead and have a PR for this independent of SLEP006, It'd be required anyway.

Labels: New Feature, module:feature_selection
Allow for passing additional parameters to the estimator's `fit` method in `SequentialFeatureSelector`

### Describe the workflow you want to enable

I would like to be able to pass sample weights to the `fit` method of the estimator in `SequentialFeatureSelector`. (`SelectFromModel` has this feature as well.)

Looking at the code, it seems to me that [`sklearn.model_selection._validation._fit_and_score`](https://github.com/scikit-learn/scikit-learn/blob/2e481f114169396660f0051eee1bcf6bcddfd556/sklearn/model_selection/_validation.py#L662), which is where the actual scoring takes place, already supports sample weights in cross-validation (i.e., takes care of passing only the weights for samples that are actually part of the fold).

### Describe your proposed solution

Based on my current understanding, `SequentialFeatureSelector.fit()` would need to get an additional argument `fit_params`  (cf. [`SelectFromModel.fit()`](https://scikit-learn.org/stable/modules/generated/sklearn.feature_selection.SelectFromModel.html#sklearn.feature_selection.SelectFromModel.fit)) that is passed on to the `cross_val_score` function called in `SequentialFeatureSelector._get_best_new_feature_score`. Everything else appears to be in place already.

if that's all it takes, I'm happy to prepare a PR. But I might very well be overlooking issues or unintended consequences.

### Describe alternatives you've considered, if relevant

_No response_

### Additional context

_No response_

COMMENT:
@adrinjalali Thanks for looking into this! Is there already a way to pass `sample_weight` to `score()`? According to SLEP006, this is not yet possible, but I might be reading this wrong.

If there's already a way to accomplish this, I'm happy to give adding a `sample_weight` parameter to `SequentialFeatureSelector`'s `fit()` a try.

Labels: New Feature, module:feature_selection
Allow for passing additional parameters to the estimator's `fit` method in `SequentialFeatureSelector`

### Describe the workflow you want to enable

I would like to be able to pass sample weights to the `fit` method of the estimator in `SequentialFeatureSelector`. (`SelectFromModel` has this feature as well.)

Looking at the code, it seems to me that [`sklearn.model_selection._validation._fit_and_score`](https://github.com/scikit-learn/scikit-learn/blob/2e481f114169396660f0051eee1bcf6bcddfd556/sklearn/model_selection/_validation.py#L662), which is where the actual scoring takes place, already supports sample weights in cross-validation (i.e., takes care of passing only the weights for samples that are actually part of the fold).

### Describe your proposed solution

Based on my current understanding, `SequentialFeatureSelector.fit()` would need to get an additional argument `fit_params`  (cf. [`SelectFromModel.fit()`](https://scikit-learn.org/stable/modules/generated/sklearn.feature_selection.SelectFromModel.html#sklearn.feature_selection.SelectFromModel.fit)) that is passed on to the `cross_val_score` function called in `SequentialFeatureSelector._get_best_new_feature_score`. Everything else appears to be in place already.

if that's all it takes, I'm happy to prepare a PR. But I might very well be overlooking issues or unintended consequences.

### Describe alternatives you've considered, if relevant

_No response_

### Additional context

_No response_

COMMENT:
No we can't support `score()` here for `sample_weight`. We can only add it as a fit param kinda thing, and pass it to `cross_val_score` as `fit_params`.

Labels: New Feature, module:feature_selection
Allow for passing additional parameters to the estimator's `fit` method in `SequentialFeatureSelector`

### Describe the workflow you want to enable

I would like to be able to pass sample weights to the `fit` method of the estimator in `SequentialFeatureSelector`. (`SelectFromModel` has this feature as well.)

Looking at the code, it seems to me that [`sklearn.model_selection._validation._fit_and_score`](https://github.com/scikit-learn/scikit-learn/blob/2e481f114169396660f0051eee1bcf6bcddfd556/sklearn/model_selection/_validation.py#L662), which is where the actual scoring takes place, already supports sample weights in cross-validation (i.e., takes care of passing only the weights for samples that are actually part of the fold).

### Describe your proposed solution

Based on my current understanding, `SequentialFeatureSelector.fit()` would need to get an additional argument `fit_params`  (cf. [`SelectFromModel.fit()`](https://scikit-learn.org/stable/modules/generated/sklearn.feature_selection.SelectFromModel.html#sklearn.feature_selection.SelectFromModel.fit)) that is passed on to the `cross_val_score` function called in `SequentialFeatureSelector._get_best_new_feature_score`. Everything else appears to be in place already.

if that's all it takes, I'm happy to prepare a PR. But I might very well be overlooking issues or unintended consequences.

### Describe alternatives you've considered, if relevant

_No response_

### Additional context

_No response_

COMMENT:
Just to be sure that I understand correctly: `sample_weight` would be considered during fitting but not during scoring, right?

If that's the case, I don't think there's much use for a `sample_weight` parameter right now. At the very least, it would be quite misleading for casual users that just look at the method's signature. I also cannot think of a case where fitting to weighted samples but scoring on unweighted samples is useful.

Labels: New Feature, module:feature_selection