Talip7/scikit-learn-issues-multilabel huggingface dataset:
RFECV regression output of cv_results_ values always of length 11 independent of n_features_

I am trying to use RFECV on a regression task, with a dataset that has a high feature to samples ratio and contains a lot of noise. As expected, the calculated value of `n_features` is quite high.

However contrary to the documentation, the ndarrays contained in `cv_results` are *not* of shape `(n_features_,)`, but consistently seem to be of length 11 no matter how much I play around with the various script parameters (such as number of folds, example dataset properties, etc.)

I re-read the documentation several times to make sure that I am using this properly. Example codes available on various sources usually present this method in a classification context, so one guess is that this function is somehow not well suited for regression tasks?!

Below is a working minimal code example that shows versions numbers of installed packages as well as the observed behaviour:
```
import sys
import numpy as np
import sklearn
from sklearn.preprocessing import MinMaxScaler
from sklearn.model_selection import KFold
from sklearn.feature_selection import RFECV
from sklearn.ensemble import RandomForestRegressor
from sklearn.datasets import make_regression

print('Python version:', sys.version)
print('Sklearn version:', sklearn.__version__)
print('Numpy version:', np.__version__)

seed=42

# approximate my real life dataset + preprocessing steps
X, y = make_regression(
    n_samples=60, n_features=1260, n_informative=250, n_targets=1, effective_rank=7, random_state=seed)
scaler_x = MinMaxScaler(feature_range=(-1, 1))
scaler_y = MinMaxScaler(feature_range=(0, 1))
X_sc = scaler_x.fit_transform(X)
y_sc = scaler_y.fit_transform(y.reshape(-1, 1))

# issue was tested on various numbers of cv folds
rf = RandomForestRegressor(n_estimators=100, n_jobs=-1, oob_score=True, bootstrap=True, random_state=seed)
kf = KFold(n_splits=5, shuffle=True, random_state=seed)

rfecv = RFECV(
    estimator=rf,
    step=0.1,
    cv=kf,
    scoring='neg_root_mean_squared_error'
)
rfecv.fit(X_sc, np.ravel(y_sc))

print('Optimal number of features :', rfecv.n_features_)
print('Length of cv_results_[mean_test_score]:', len(rfecv.cv_results_['mean_test_score']))
```

Output:

Python version: 3.10.4 | packaged by conda-forge | (main, Mar 30 2022, 08:38:02) [MSC v.1916 64 bit (AMD64)]
Sklearn version: 1.1.1
Numpy version: 1.22.4
Optimal number of features : 252
Length of cv_results_[mean_test_score]: 11

COMMENT:
Hi,
thanks for the clarification. I am unsure what a PR means or, subsequently,  how to open one. I presume you mean opening a new issue and asking for documentation updates?

I found the note in the documentation explaining how this value was calculated for `grid_scores_`. Seeing that, there also seems to be some level of confusion going on in the documentation between `n_features`, `n_features_` and `n_features_in_`.

Kindly let me know how to best proceed & have a nice day!

Labels: Documentation, module:feature_selection
RFECV regression output of cv_results_ values always of length 11 independent of n_features_

I am trying to use RFECV on a regression task, with a dataset that has a high feature to samples ratio and contains a lot of noise. As expected, the calculated value of `n_features` is quite high.

However contrary to the documentation, the ndarrays contained in `cv_results` are *not* of shape `(n_features_,)`, but consistently seem to be of length 11 no matter how much I play around with the various script parameters (such as number of folds, example dataset properties, etc.)

I re-read the documentation several times to make sure that I am using this properly. Example codes available on various sources usually present this method in a classification context, so one guess is that this function is somehow not well suited for regression tasks?!

Below is a working minimal code example that shows versions numbers of installed packages as well as the observed behaviour:
```
import sys
import numpy as np
import sklearn
from sklearn.preprocessing import MinMaxScaler
from sklearn.model_selection import KFold
from sklearn.feature_selection import RFECV
from sklearn.ensemble import RandomForestRegressor
from sklearn.datasets import make_regression

print('Python version:', sys.version)
print('Sklearn version:', sklearn.__version__)
print('Numpy version:', np.__version__)

seed=42

# approximate my real life dataset + preprocessing steps
X, y = make_regression(
    n_samples=60, n_features=1260, n_informative=250, n_targets=1, effective_rank=7, random_state=seed)
scaler_x = MinMaxScaler(feature_range=(-1, 1))
scaler_y = MinMaxScaler(feature_range=(0, 1))
X_sc = scaler_x.fit_transform(X)
y_sc = scaler_y.fit_transform(y.reshape(-1, 1))

# issue was tested on various numbers of cv folds
rf = RandomForestRegressor(n_estimators=100, n_jobs=-1, oob_score=True, bootstrap=True, random_state=seed)
kf = KFold(n_splits=5, shuffle=True, random_state=seed)

rfecv = RFECV(
    estimator=rf,
    step=0.1,
    cv=kf,
    scoring='neg_root_mean_squared_error'
)
rfecv.fit(X_sc, np.ravel(y_sc))

print('Optimal number of features :', rfecv.n_features_)
print('Length of cv_results_[mean_test_score]:', len(rfecv.cv_results_['mean_test_score']))
```

Output:

Python version: 3.10.4 | packaged by conda-forge | (main, Mar 30 2022, 08:38:02) [MSC v.1916 64 bit (AMD64)]
Sklearn version: 1.1.1
Numpy version: 1.22.4
Optimal number of features : 252
Length of cv_results_[mean_test_score]: 11

COMMENT:
Ah my bad, I'm not fully awake yet. We're talking about a pull request.
I shall open one and correct the few lines of documentation to say `n_subsets_of_features`. I will leave this issue open for now though, maybe someone can clear up the `n_features[_][in_]` uncertainty.

Labels: Documentation, module:feature_selection
RFECV regression output of cv_results_ values always of length 11 independent of n_features_

I am trying to use RFECV on a regression task, with a dataset that has a high feature to samples ratio and contains a lot of noise. As expected, the calculated value of `n_features` is quite high.

However contrary to the documentation, the ndarrays contained in `cv_results` are *not* of shape `(n_features_,)`, but consistently seem to be of length 11 no matter how much I play around with the various script parameters (such as number of folds, example dataset properties, etc.)

I re-read the documentation several times to make sure that I am using this properly. Example codes available on various sources usually present this method in a classification context, so one guess is that this function is somehow not well suited for regression tasks?!

Below is a working minimal code example that shows versions numbers of installed packages as well as the observed behaviour:
```
import sys
import numpy as np
import sklearn
from sklearn.preprocessing import MinMaxScaler
from sklearn.model_selection import KFold
from sklearn.feature_selection import RFECV
from sklearn.ensemble import RandomForestRegressor
from sklearn.datasets import make_regression

print('Python version:', sys.version)
print('Sklearn version:', sklearn.__version__)
print('Numpy version:', np.__version__)

seed=42

# approximate my real life dataset + preprocessing steps
X, y = make_regression(
    n_samples=60, n_features=1260, n_informative=250, n_targets=1, effective_rank=7, random_state=seed)
scaler_x = MinMaxScaler(feature_range=(-1, 1))
scaler_y = MinMaxScaler(feature_range=(0, 1))
X_sc = scaler_x.fit_transform(X)
y_sc = scaler_y.fit_transform(y.reshape(-1, 1))

# issue was tested on various numbers of cv folds
rf = RandomForestRegressor(n_estimators=100, n_jobs=-1, oob_score=True, bootstrap=True, random_state=seed)
kf = KFold(n_splits=5, shuffle=True, random_state=seed)

rfecv = RFECV(
    estimator=rf,
    step=0.1,
    cv=kf,
    scoring='neg_root_mean_squared_error'
)
rfecv.fit(X_sc, np.ravel(y_sc))

print('Optimal number of features :', rfecv.n_features_)
print('Length of cv_results_[mean_test_score]:', len(rfecv.cv_results_['mean_test_score']))
```

Output:

Python version: 3.10.4 | packaged by conda-forge | (main, Mar 30 2022, 08:38:02) [MSC v.1916 64 bit (AMD64)]
Sklearn version: 1.1.1
Numpy version: 1.22.4
Optimal number of features : 252
Length of cv_results_[mean_test_score]: 11

COMMENT:
`n_features`, `n_features_` and `n_features_in_` are all the same. `n_features_in_` was added later to all transformers to have a consistent API. The "in" was added to distinguish between "input" features and "output" features.

Labels: Documentation, module:feature_selection
RFECV regression output of cv_results_ values always of length 11 independent of n_features_

I am trying to use RFECV on a regression task, with a dataset that has a high feature to samples ratio and contains a lot of noise. As expected, the calculated value of `n_features` is quite high.

However contrary to the documentation, the ndarrays contained in `cv_results` are *not* of shape `(n_features_,)`, but consistently seem to be of length 11 no matter how much I play around with the various script parameters (such as number of folds, example dataset properties, etc.)

I re-read the documentation several times to make sure that I am using this properly. Example codes available on various sources usually present this method in a classification context, so one guess is that this function is somehow not well suited for regression tasks?!

Below is a working minimal code example that shows versions numbers of installed packages as well as the observed behaviour:
```
import sys
import numpy as np
import sklearn
from sklearn.preprocessing import MinMaxScaler
from sklearn.model_selection import KFold
from sklearn.feature_selection import RFECV
from sklearn.ensemble import RandomForestRegressor
from sklearn.datasets import make_regression

print('Python version:', sys.version)
print('Sklearn version:', sklearn.__version__)
print('Numpy version:', np.__version__)

seed=42

# approximate my real life dataset + preprocessing steps
X, y = make_regression(
    n_samples=60, n_features=1260, n_informative=250, n_targets=1, effective_rank=7, random_state=seed)
scaler_x = MinMaxScaler(feature_range=(-1, 1))
scaler_y = MinMaxScaler(feature_range=(0, 1))
X_sc = scaler_x.fit_transform(X)
y_sc = scaler_y.fit_transform(y.reshape(-1, 1))

# issue was tested on various numbers of cv folds
rf = RandomForestRegressor(n_estimators=100, n_jobs=-1, oob_score=True, bootstrap=True, random_state=seed)
kf = KFold(n_splits=5, shuffle=True, random_state=seed)

rfecv = RFECV(
    estimator=rf,
    step=0.1,
    cv=kf,
    scoring='neg_root_mean_squared_error'
)
rfecv.fit(X_sc, np.ravel(y_sc))

print('Optimal number of features :', rfecv.n_features_)
print('Length of cv_results_[mean_test_score]:', len(rfecv.cv_results_['mean_test_score']))
```

Output:

Python version: 3.10.4 | packaged by conda-forge | (main, Mar 30 2022, 08:38:02) [MSC v.1916 64 bit (AMD64)]
Sklearn version: 1.1.1
Numpy version: 1.22.4
Optimal number of features : 252
Length of cv_results_[mean_test_score]: 11

COMMENT:
Thank you for the clarification!

I looked into updating the documentation and opening a pull request. Unfortunately this requires a number of tools that I do not have available and am not allowed to install on my (company issued) computer. I am very sorry, I will have to leave fixing this issue for someone else.

Labels: Documentation, module:feature_selection
RFECV regression output of cv_results_ values always of length 11 independent of n_features_

I am trying to use RFECV on a regression task, with a dataset that has a high feature to samples ratio and contains a lot of noise. As expected, the calculated value of `n_features` is quite high.

However contrary to the documentation, the ndarrays contained in `cv_results` are *not* of shape `(n_features_,)`, but consistently seem to be of length 11 no matter how much I play around with the various script parameters (such as number of folds, example dataset properties, etc.)

I re-read the documentation several times to make sure that I am using this properly. Example codes available on various sources usually present this method in a classification context, so one guess is that this function is somehow not well suited for regression tasks?!

Below is a working minimal code example that shows versions numbers of installed packages as well as the observed behaviour:
```
import sys
import numpy as np
import sklearn
from sklearn.preprocessing import MinMaxScaler
from sklearn.model_selection import KFold
from sklearn.feature_selection import RFECV
from sklearn.ensemble import RandomForestRegressor
from sklearn.datasets import make_regression

print('Python version:', sys.version)
print('Sklearn version:', sklearn.__version__)
print('Numpy version:', np.__version__)

seed=42

# approximate my real life dataset + preprocessing steps
X, y = make_regression(
    n_samples=60, n_features=1260, n_informative=250, n_targets=1, effective_rank=7, random_state=seed)
scaler_x = MinMaxScaler(feature_range=(-1, 1))
scaler_y = MinMaxScaler(feature_range=(0, 1))
X_sc = scaler_x.fit_transform(X)
y_sc = scaler_y.fit_transform(y.reshape(-1, 1))

# issue was tested on various numbers of cv folds
rf = RandomForestRegressor(n_estimators=100, n_jobs=-1, oob_score=True, bootstrap=True, random_state=seed)
kf = KFold(n_splits=5, shuffle=True, random_state=seed)

rfecv = RFECV(
    estimator=rf,
    step=0.1,
    cv=kf,
    scoring='neg_root_mean_squared_error'
)
rfecv.fit(X_sc, np.ravel(y_sc))

print('Optimal number of features :', rfecv.n_features_)
print('Length of cv_results_[mean_test_score]:', len(rfecv.cv_results_['mean_test_score']))
```

Output:

Python version: 3.10.4 | packaged by conda-forge | (main, Mar 30 2022, 08:38:02) [MSC v.1916 64 bit (AMD64)]
Sklearn version: 1.1.1
Numpy version: 1.22.4
Optimal number of features : 252
Length of cv_results_[mean_test_score]: 11

COMMENT:
> The documentation for `cv_results_` is incorrect. It should be `n_subsets_of_features`.

Working on this

Labels: Documentation, module:feature_selection
Compute the gradient norm of t-SNE only when checking convergence

### Describe the workflow you want to enable

Inside the  [`_gradient_descent()` function](https://github.com/scikit-learn/scikit-learn/blob/80598905e517759b4696c74ecc35c6e2eb508cff/sklearn/manifold/_t_sne.py#L399) in `_t_sne.py`, a gradient norm is computed for every iteration

```
grad_norm = linalg.norm(grad)
```

But its magnitude checking is performed only every `n_iter_check` iterations.
It is therefore possible to compute norm less frequently to same some computations.

### Describe your proposed solution

Move the line `grad_norm = linalg.norm(grad)` to a later part, immediately before the line

```
if grad_norm <= min_grad_norm:
```
So if other early-stopping criteria are met, no need to compute norm.

### Describe alternatives you've considered, if relevant

_No response_

### Additional context

_No response_

COMMENT:
`grad_norm` is indeed only used for convergence checking. You're right, it should be moved into the `if check_convergence` block. Would you be interested in making a PR @Xunius ?

Labels: module:manifold
TF*IDF yields different results than TfidfTransformer

### Describe the bug

I was trying out some TF-IDF for NLP with sklearn. To double check the results, I was expecting to get the same output tfidf_matrix == TF*IDF where TF is the output of the `CountVectorizer` and IDF is the`tfidf_transformer.idf_` I was suprise to see both arrays are not the same (returns `False`):

<img width="1280" alt="image" src="https://user-images.githubusercontent.com/17580456/171156940-6455c8d8-b61c-45c2-a1ac-f1fba11d6965.png">
<img width="1280" alt="image" src="https://user-images.githubusercontent.com/17580456/171157032-3c519817-92e2-4abc-9b74-4449d390120b.png">

Is this a bug or I am missing something silly? 



### Steps/Code to Reproduce

-

### Expected Results

Both arrays to be the same

### Actual Results

Results differ

### Versions

```shell
System:
    python: 3.7.13 (default, Apr 24 2022, 01:04:09)  [GCC 7.5.0]
executable: /usr/bin/python3
   machine: Linux-5.4.188+-x86_64-with-Ubuntu-18.04-bionic

Python dependencies:
          pip: 21.1.3
   setuptools: 57.4.0
      sklearn: 1.0.2
        numpy: 1.21.6
        scipy: 1.4.1
       Cython: 0.29.30
       pandas: 1.3.5
   matplotlib: 3.2.2
       joblib: 1.1.0
threadpoolctl: 3.1.0

Built with OpenMP: True
```

COMMENT:
At transform, `TfidfTransformer` normalizes the vectors, hence the difference you observe. Both will be equal if you set `norm=None` in `TfidfTransformer`.

This is not clearly explained in the docstring (and `None` is not even listed as a valid entry for the `norm` parameter). Would you be interested in submitting a PR to improve the doc @gloriamacia ?

Labels: Documentation, module:feature_extraction
TF*IDF yields different results than TfidfTransformer

### Describe the bug

I was trying out some TF-IDF for NLP with sklearn. To double check the results, I was expecting to get the same output tfidf_matrix == TF*IDF where TF is the output of the `CountVectorizer` and IDF is the`tfidf_transformer.idf_` I was suprise to see both arrays are not the same (returns `False`):

<img width="1280" alt="image" src="https://user-images.githubusercontent.com/17580456/171156940-6455c8d8-b61c-45c2-a1ac-f1fba11d6965.png">
<img width="1280" alt="image" src="https://user-images.githubusercontent.com/17580456/171157032-3c519817-92e2-4abc-9b74-4449d390120b.png">

Is this a bug or I am missing something silly? 



### Steps/Code to Reproduce

-

### Expected Results

Both arrays to be the same

### Actual Results

Results differ

### Versions

```shell
System:
    python: 3.7.13 (default, Apr 24 2022, 01:04:09)  [GCC 7.5.0]
executable: /usr/bin/python3
   machine: Linux-5.4.188+-x86_64-with-Ubuntu-18.04-bionic

Python dependencies:
          pip: 21.1.3
   setuptools: 57.4.0
      sklearn: 1.0.2
        numpy: 1.21.6
        scipy: 1.4.1
       Cython: 0.29.30
       pandas: 1.3.5
   matplotlib: 3.2.2
       joblib: 1.1.0
threadpoolctl: 3.1.0

Built with OpenMP: True
```

COMMENT:
@jeremiedbb I can confirm, `norm=None` makes both results equal as I expected: 

<img width="967" alt="image" src="https://user-images.githubusercontent.com/17580456/172045890-60289eb7-2488-4694-ada3-dd6f3646fe72.png">

I then explored the `norm` argument and I figured out the following: 

`norm` can take `None` or` False` in which case no [normalization](https://scikit-learn.org/stable/modules/generated/sklearn.preprocessing.normalize.html) is applied. On the other hand, `norm `can also take `True` in which case `l2` normalization is applied. Finally, `norm `can also take as arguments `l2` or `l1` which make explicit the type of normalization being used. 

<img width="967" alt="image" src="https://user-images.githubusercontent.com/17580456/172046297-b195b51e-2f06-4025-a874-a1e097abdff8.png">

I personally feel `True` or `False `should not be supported as they prone to user errors, only `None`, `l1 `or `l2`. This could be a transformer improvement. For now, as you suggested I will focus on improving the documentation which I found [here](https://github.com/scikit-learn/scikit-learn/blob/3578a9f9825514d3bfd2f2448bf4e0f95326b938/sklearn/feature_extraction/text.py). 

Where can I find this [documentation page](https://scikit-learn.org/stable/modules/generated/sklearn.feature_extraction.text.TfidfTransformer.html) as code to edit it? 

I looked for in it in vain in the [feature_extraction folder ](https://github.com/scikit-learn/scikit-learn/tree/3578a9f9825514d3bfd2f2448bf4e0f95326b938/sklearn/feature_extraction)

Labels: Documentation, module:feature_extraction
TF*IDF yields different results than TfidfTransformer

### Describe the bug

I was trying out some TF-IDF for NLP with sklearn. To double check the results, I was expecting to get the same output tfidf_matrix == TF*IDF where TF is the output of the `CountVectorizer` and IDF is the`tfidf_transformer.idf_` I was suprise to see both arrays are not the same (returns `False`):

<img width="1280" alt="image" src="https://user-images.githubusercontent.com/17580456/171156940-6455c8d8-b61c-45c2-a1ac-f1fba11d6965.png">
<img width="1280" alt="image" src="https://user-images.githubusercontent.com/17580456/171157032-3c519817-92e2-4abc-9b74-4449d390120b.png">

Is this a bug or I am missing something silly? 



### Steps/Code to Reproduce

-

### Expected Results

Both arrays to be the same

### Actual Results

Results differ

### Versions

```shell
System:
    python: 3.7.13 (default, Apr 24 2022, 01:04:09)  [GCC 7.5.0]
executable: /usr/bin/python3
   machine: Linux-5.4.188+-x86_64-with-Ubuntu-18.04-bionic

Python dependencies:
          pip: 21.1.3
   setuptools: 57.4.0
      sklearn: 1.0.2
        numpy: 1.21.6
        scipy: 1.4.1
       Cython: 0.29.30
       pandas: 1.3.5
   matplotlib: 3.2.2
       joblib: 1.1.0
threadpoolctl: 3.1.0

Built with OpenMP: True
```

COMMENT:
found it! https://github.com/scikit-learn/scikit-learn/blob/80598905e/sklearn/feature_extraction/text.py#L1469

Labels: Documentation, module:feature_extraction
TF*IDF yields different results than TfidfTransformer

### Describe the bug

I was trying out some TF-IDF for NLP with sklearn. To double check the results, I was expecting to get the same output tfidf_matrix == TF*IDF where TF is the output of the `CountVectorizer` and IDF is the`tfidf_transformer.idf_` I was suprise to see both arrays are not the same (returns `False`):

<img width="1280" alt="image" src="https://user-images.githubusercontent.com/17580456/171156940-6455c8d8-b61c-45c2-a1ac-f1fba11d6965.png">
<img width="1280" alt="image" src="https://user-images.githubusercontent.com/17580456/171157032-3c519817-92e2-4abc-9b74-4449d390120b.png">

Is this a bug or I am missing something silly? 



### Steps/Code to Reproduce

-

### Expected Results

Both arrays to be the same

### Actual Results

Results differ

### Versions

```shell
System:
    python: 3.7.13 (default, Apr 24 2022, 01:04:09)  [GCC 7.5.0]
executable: /usr/bin/python3
   machine: Linux-5.4.188+-x86_64-with-Ubuntu-18.04-bionic

Python dependencies:
          pip: 21.1.3
   setuptools: 57.4.0
      sklearn: 1.0.2
        numpy: 1.21.6
        scipy: 1.4.1
       Cython: 0.29.30
       pandas: 1.3.5
   matplotlib: 3.2.2
       joblib: 1.1.0
threadpoolctl: 3.1.0

Built with OpenMP: True
```

COMMENT:
> On the other hand, norm can also take True in which case l2 normalization is applied. Finally, norm can also take as arguments l2 or l1 which make explicit the type of normalization being used.

I don't think norm can be `True`. In that case, an error is raised during normalization.

I agree that True/False should not be supported. The only thing we should do to document that `None` is a possibility (and what it means) in the docstring

Labels: Documentation, module:feature_extraction