Talip7/scikit-learn-issues-multilabel huggingface dataset:
StackingClassifier crashes with stack_method="predict"

### Describe the bug

I trained a StackingClassifier with 8 estimators and logistic regression final estimator, with stacking method of "predict" and after the fit, when i try to do predict i get this error:

Traceback (most recent call last):
  File "<string>", line 1, in <module>
  File "/root/anaconda3/lib/python3.9/site-packages/sklearn/ensemble/_stacking.py", line 627, in predict_proba
    return self.final_estimator_.predict_proba(self.transform(X))
  File "/root/anaconda3/lib/python3.9/site-packages/sklearn/ensemble/_stacking.py", line 663, in transform
    return self._transform(X)
  File "/root/anaconda3/lib/python3.9/site-packages/sklearn/ensemble/_stacking.py", line 281, in _transform
    return self._concatenate_predictions(X, predictions)
  File "/root/anaconda3/lib/python3.9/site-packages/sklearn/ensemble/_stacking.py", line 99, in _concatenate_predictions
    X_meta.append(preds.reshape(-1, 1))
  File "/root/anaconda3/lib/python3.9/site-packages/pandas/core/generic.py", line 5487, in __getattr__
    return object.__getattribute__(self, name)
AttributeError: 'Series' object has no attribute 'reshape'

### Steps/Code to Reproduce

mega_model.fit(data, y)
res = mega_model.predict(data)

### Expected Results

an array of predictions

### Actual Results

the error

### Versions

```shell
System:
    python: 3.9.7 (default, Sep 16 2021, 13:09:58)  [GCC 7.5.0]
executable: /root/anaconda3/bin/python
   machine: Linux-4.18.0-408.el8.x86_64-x86_64-with-glibc2.28

Python dependencies:
      sklearn: 1.1.2
          pip: 22.2.2
   setuptools: 58.0.4
        numpy: 1.19.5
        scipy: 1.7.1
       Cython: 0.29.24
       pandas: 1.3.4
   matplotlib: 3.4.3
       joblib: 1.1.0
threadpoolctl: 2.2.0

Built with OpenMP: True

threadpoolctl info:
       filepath: /root/anaconda3/lib/python3.9/site-packages/scikit_learn.libs/libgomp-a34b3233.so.1.0.0
         prefix: libgomp
       user_api: openmp
   internal_api: openmp
        version: None
    num_threads: 32

       filepath: /root/anaconda3/lib/python3.9/site-packages/numpy.libs/libopenblasp-r0-09e95953.3.13.so
         prefix: libopenblas
       user_api: blas
   internal_api: openblas
        version: 0.3.13
    num_threads: 32
threading_layer: pthreads
   architecture: Sandybridge

       filepath: /root/anaconda3/lib/libmkl_rt.so.1
         prefix: libmkl_rt
       user_api: blas
   internal_api: mkl
        version: 2021.4-Product
    num_threads: 16
threading_layer: intel

       filepath: /root/anaconda3/lib/libiomp5.so
         prefix: libiomp
       user_api: openmp
   internal_api: openmp
        version: None
    num_threads: 32
```

COMMENT:
it was my bug

Labels: Bug, Needs Triage
Example performs data preprocessing before train-test split occurs

### Describe the issue linked to the documentation

[This example](https://scikit-learn.org/stable/auto_examples/classification/plot_classifier_comparison.html#sphx-glr-auto-examples-classification-plot-classifier-comparison-py) uses the following code 



    X = StandardScaler().fit_transform(X)

    X_train, X_test, y_train, y_test = train_test_split(
        X, y, test_size=0.4, random_state=42
    )

As can be seen, the data preprocessing occurs before the data is split into a train and test set.
It might be insignificant in this particular case. Still, inexperienced people will likely be inclined to copy this example and make the same mistake on a larger scale, potentially resulting in harmful real-world consequences.




### Suggest a potential alternative/fix

Consequently, it might be advisable to replace this code with the following

    X_train, X_test, y_train, y_test = train_test_split(
        X, y, test_size=0.4, random_state=42
    )
    SS = StandardScaler()
    X_train = SS.fit_transform(X_train)
    X_test = SS.transform(X_test)

COMMENT:
I don't think that we should be using the estimator API here. If we use the estimator API then, we should show the usage of the `Pipeline` directly.

I would rather add a small block of code that could be part of the data generation that uses `sklearn.preprocessing.scale` with an explicit comment that for simplicity we use it but in practice, one needs to define a ML model where the preprocessing is part of this pipeline.

Labels: Documentation, Needs Triage
Example performs data preprocessing before train-test split occurs

### Describe the issue linked to the documentation

[This example](https://scikit-learn.org/stable/auto_examples/classification/plot_classifier_comparison.html#sphx-glr-auto-examples-classification-plot-classifier-comparison-py) uses the following code 



    X = StandardScaler().fit_transform(X)

    X_train, X_test, y_train, y_test = train_test_split(
        X, y, test_size=0.4, random_state=42
    )

As can be seen, the data preprocessing occurs before the data is split into a train and test set.
It might be insignificant in this particular case. Still, inexperienced people will likely be inclined to copy this example and make the same mistake on a larger scale, potentially resulting in harmful real-world consequences.




### Suggest a potential alternative/fix

Consequently, it might be advisable to replace this code with the following

    X_train, X_test, y_train, y_test = train_test_split(
        X, y, test_size=0.4, random_state=42
    )
    SS = StandardScaler()
    X_train = SS.fit_transform(X_train)
    X_test = SS.transform(X_test)

COMMENT:
Maybe we could stick to a pipeline by just wrapping the `classifiers` list?

```py
classifiers = [make_pipeline(StandardScaler(), clf) for clf in classifiers]
```

Labels: Documentation, Needs Triage
Example performs data preprocessing before train-test split occurs

### Describe the issue linked to the documentation

[This example](https://scikit-learn.org/stable/auto_examples/classification/plot_classifier_comparison.html#sphx-glr-auto-examples-classification-plot-classifier-comparison-py) uses the following code 



    X = StandardScaler().fit_transform(X)

    X_train, X_test, y_train, y_test = train_test_split(
        X, y, test_size=0.4, random_state=42
    )

As can be seen, the data preprocessing occurs before the data is split into a train and test set.
It might be insignificant in this particular case. Still, inexperienced people will likely be inclined to copy this example and make the same mistake on a larger scale, potentially resulting in harmful real-world consequences.




### Suggest a potential alternative/fix

Consequently, it might be advisable to replace this code with the following

    X_train, X_test, y_train, y_test = train_test_split(
        X, y, test_size=0.4, random_state=42
    )
    SS = StandardScaler()
    X_train = SS.fit_transform(X_train)
    X_test = SS.transform(X_test)

COMMENT:
Why not. Looking a bit more at the example, we could simplify using a dict and have a readable name without too much of a hassle.

```python
classifiers = {name: make_pipeline(StandardScaler(), clf) for name, clf in zip(name, classifiers)}
```

Labels: Documentation, Needs Triage
Deprecate n_features_ in PCA

When n_features_in_ was introduced for all estimators, some estimators already had a `n_features_` attribute that was deprecated in favor of n_features_in_. It should have been done for PCA and RFE as well, but we must have missed it. We need to deprecate in 1.2 for removal in 1.4 now.

COMMENT:
Hi @jeremiedbb , can I work on this issue?

Labels: API
Deprecate n_features_ in PCA

When n_features_in_ was introduced for all estimators, some estimators already had a `n_features_` attribute that was deprecated in favor of n_features_in_. It should have been done for PCA and RFE as well, but we must have missed it. We need to deprecate in 1.2 for removal in 1.4 now.

COMMENT:
Sure, go ahead !

Labels: API
Deprecate n_features_ in PCA

When n_features_in_ was introduced for all estimators, some estimators already had a `n_features_` attribute that was deprecated in favor of n_features_in_. It should have been done for PCA and RFE as well, but we must have missed it. We need to deprecate in 1.2 for removal in 1.4 now.

COMMENT:
We should have a separate PR for these 2 estimators. For `RFE`, I think that we should rename the attribute because it is a misleading name. It should be something like `n_features_selected_` that is different from `n_features_in_`. Indeed, it would correspond to `n_features_out_`.

Labels: API
Deprecate n_features_ in PCA

When n_features_in_ was introduced for all estimators, some estimators already had a `n_features_` attribute that was deprecated in favor of n_features_in_. It should have been done for PCA and RFE as well, but we must have missed it. We need to deprecate in 1.2 for removal in 1.4 now.

COMMENT:
As pointed out by @glemaitre, n_features_ in RFE is not equivalent to n_features_in_, so let's deprecate it only in PCA for now. we'll deal with RFE when introducing n_features_out_

Labels: API
Deprecate n_features_ in PCA

When n_features_in_ was introduced for all estimators, some estimators already had a `n_features_` attribute that was deprecated in favor of n_features_in_. It should have been done for PCA and RFE as well, but we must have missed it. We need to deprecate in 1.2 for removal in 1.4 now.

COMMENT:
Hi @jeremiedbb @glemaitre ,
I have picked up this issue and raised PR

Labels: API
DOC: new example on feature engineering for time-series forecasting with prediction intervals

For EuroScipy 2022, I gave a tutorial on how to use pandas-engineered lagged and windowed features for time series forecasting with scikit-learn regressors.

Here is the notebook:

- https://nbviewer.org/github/ogrisel/euroscipy-2022-time-series/blob/main/plot_time_series_feature_engineering.ipynb

I think it might be worth investing some effort to reuse some of that material to turn it into a tutorialish example for the gallery (and cross-link it with the time-related feature example on the same dataset).

Note that this example probably covers too many thinks and we should probably focus on a trim down version. For instance, by removing the experiment with MAPIE that I don't find particularly conclusive at the moment (would need to speed more time to find how to make MAPIE output heteroscedastic prediction intervals on this data in particular). I find that the discussion on sktime to be informative to go beyond the pure-scikit-learn approach which I found out to be limitting in retrospect as explained towards the end of the tutorial.

/cc @lorentzenchr who expressed interest.
Also /cc @ArturoAmorQ who might be interested in working on such a contribution.

COMMENT:
Hey @ogrisel, I went through the tutorial notebook you provided and the scikit-learn example on [Time-related Feature Engineering](https://scikit-learn.org/stable/auto_examples/applications/plot_cyclical_feature_engineering.html) using the bike sharing dataset.

> Note that this example probably covers too many thinks and we should probably focus on a trim down version.

Based on the code in the example, I created a trimmed down version of it that makes use of pandas-engineered lagged features and Histogram-based Gradient Boosting Regression Trees for time-series forecasting over [HERE](https://colab.research.google.com/drive/1mhvqNuXXBCm9Cp_wHulJ4mIHPEqOuo1s#scrollTo=PXyLv6esVTMj).

Can I take up this issue and make a PR to include code from the trimmed down version? We can then cross-link it to the time-related feature example on the same dataset.

Labels: Documentation
DOC: new example on feature engineering for time-series forecasting with prediction intervals

For EuroScipy 2022, I gave a tutorial on how to use pandas-engineered lagged and windowed features for time series forecasting with scikit-learn regressors.

Here is the notebook:

- https://nbviewer.org/github/ogrisel/euroscipy-2022-time-series/blob/main/plot_time_series_feature_engineering.ipynb

I think it might be worth investing some effort to reuse some of that material to turn it into a tutorialish example for the gallery (and cross-link it with the time-related feature example on the same dataset).

Note that this example probably covers too many thinks and we should probably focus on a trim down version. For instance, by removing the experiment with MAPIE that I don't find particularly conclusive at the moment (would need to speed more time to find how to make MAPIE output heteroscedastic prediction intervals on this data in particular). I find that the discussion on sktime to be informative to go beyond the pure-scikit-learn approach which I found out to be limitting in retrospect as explained towards the end of the tutorial.

/cc @lorentzenchr who expressed interest.
Also /cc @ArturoAmorQ who might be interested in working on such a contribution.

COMMENT:
@ogrisel I‘m a bit surprised that you merged without any review.

Labels: Documentation
DOC: new example on feature engineering for time-series forecasting with prediction intervals

For EuroScipy 2022, I gave a tutorial on how to use pandas-engineered lagged and windowed features for time series forecasting with scikit-learn regressors.

Here is the notebook:

- https://nbviewer.org/github/ogrisel/euroscipy-2022-time-series/blob/main/plot_time_series_feature_engineering.ipynb

I think it might be worth investing some effort to reuse some of that material to turn it into a tutorialish example for the gallery (and cross-link it with the time-related feature example on the same dataset).

Note that this example probably covers too many thinks and we should probably focus on a trim down version. For instance, by removing the experiment with MAPIE that I don't find particularly conclusive at the moment (would need to speed more time to find how to make MAPIE output heteroscedastic prediction intervals on this data in particular). I find that the discussion on sktime to be informative to go beyond the pure-scikit-learn approach which I found out to be limitting in retrospect as explained towards the end of the tutorial.

/cc @lorentzenchr who expressed interest.
Also /cc @ArturoAmorQ who might be interested in working on such a contribution.

COMMENT:
Sorry i did not know you were planning to review. It's true that i was the original author of the source notebook, but since it was a long time ago and it was reworked by an external contributor and reviewed again by arturo and I thought it was enough, especially since there is no impact on the library itself and we can improve it incrementally or completely rework it a posteriori without backward compat concerns.

Labels: Documentation
DOC: new example on feature engineering for time-series forecasting with prediction intervals

For EuroScipy 2022, I gave a tutorial on how to use pandas-engineered lagged and windowed features for time series forecasting with scikit-learn regressors.

Here is the notebook:

- https://nbviewer.org/github/ogrisel/euroscipy-2022-time-series/blob/main/plot_time_series_feature_engineering.ipynb

I think it might be worth investing some effort to reuse some of that material to turn it into a tutorialish example for the gallery (and cross-link it with the time-related feature example on the same dataset).

Note that this example probably covers too many thinks and we should probably focus on a trim down version. For instance, by removing the experiment with MAPIE that I don't find particularly conclusive at the moment (would need to speed more time to find how to make MAPIE output heteroscedastic prediction intervals on this data in particular). I find that the discussion on sktime to be informative to go beyond the pure-scikit-learn approach which I found out to be limitting in retrospect as explained towards the end of the tutorial.

/cc @lorentzenchr who expressed interest.
Also /cc @ArturoAmorQ who might be interested in working on such a contribution.

COMMENT:
Sorry also from me, I confused the issue with the PR which as 2 reviews. So everything is fine.

Labels: Documentation
DOC Clarify documentation writing guideline

### Describe the issue linked to the documentation

The ['Guidelines for writing documentation'](https://scikit-learn.org/dev/developers/contributing.html#guidelines-for-writing-documentation) section seems to be specifically about docstrings and the suggestions don't seem to be as relevant for other types of documentation, e.g., example, tutorial, usage pages. 

### Suggest a potential alternative/fix

Clarify that this section is about docstrings. Potentially add sections on guidelines for writing examples, usage and tutorial pages?

COMMENT:
I don't think this is limited to docstrings. What makes you think so?

Labels: Documentation
DOC Clarify documentation writing guideline

### Describe the issue linked to the documentation

The ['Guidelines for writing documentation'](https://scikit-learn.org/dev/developers/contributing.html#guidelines-for-writing-documentation) section seems to be specifically about docstrings and the suggestions don't seem to be as relevant for other types of documentation, e.g., example, tutorial, usage pages. 

### Suggest a potential alternative/fix

Clarify that this section is about docstrings. Potentially add sections on guidelines for writing examples, usage and tutorial pages?

COMMENT:
Fair, reading it again I can see how it has more general tips than I originally thought, though maybe not necessarily relevant for all types of documentation.

I think the problem may better be described as: it seems very prescriptive, i.e., too specific in detailing exactly what to include and the order, e.g.
1. hand-waving explanation
2. why method is useful and when it should be used
3. figure
4. code examples
5. math and equations

This may not be so relevant for all types of documentation, e.g., docstrings don't usually contain figures.

I think this issue should be more - make the guide more general for all types of documentation and suggest what could be included in various types of documentation (instead of suggesting that all types of documentation should include all 5 items), but I am also happy to close this issue as this is not as much of a problem as I originally thought.

Labels: Documentation
DOC Clarify documentation writing guideline

### Describe the issue linked to the documentation

The ['Guidelines for writing documentation'](https://scikit-learn.org/dev/developers/contributing.html#guidelines-for-writing-documentation) section seems to be specifically about docstrings and the suggestions don't seem to be as relevant for other types of documentation, e.g., example, tutorial, usage pages. 

### Suggest a potential alternative/fix

Clarify that this section is about docstrings. Potentially add sections on guidelines for writing examples, usage and tutorial pages?

COMMENT:
closing this as it seems the docstring section has been updated and the above guidelines is specified for the user guide, which makes sense.

Labels: Documentation
Deprecated is_first_col() function

https://github.com/scikit-learn/scikit-learn/blob/36958fb240fbe435673a9e3c52e769f01f36bec0/examples/decomposition/plot_varimax_fa.py#L72

The is_first_col function was deprecated in Matplotlib 3.4 #40714 (https://github.com/pandas-dev/pandas/issues/40714). Could you change the function by get_subplotspec().is_first_col() instead? Thanks!

Josu

COMMENT:
I agree that we should update the example. Do you want to open a pull request with your suggestion?

Labels: Documentation
DOC deprecate is updating a docstring against numpydocs rules

### Describe the issue linked to the documentation

`sklearn.utils.deprecate` is updating the docstring within `_update_doc()`
This causes the numpydocs to fail during the test with the error:

`GL09: Deprecation warning should precede extended summary`

After discussing with @glemaitre and @ogrisel : the `_update_doc()` should be removed from `sklearn.utils.deprecate` ensuring that the appropriate depracation message appears in all the docstrings using the `@deprecated` decorator at the moment.


### Suggest a potential alternative/fix

_No response_

COMMENT:
I agree. This needs a manual inspection of all occurrences to make sure that the deprecated parameters are correctly documented though.

Labels: Documentation, Build / CI
DOC deprecate is updating a docstring against numpydocs rules

### Describe the issue linked to the documentation

`sklearn.utils.deprecate` is updating the docstring within `_update_doc()`
This causes the numpydocs to fail during the test with the error:

`GL09: Deprecation warning should precede extended summary`

After discussing with @glemaitre and @ogrisel : the `_update_doc()` should be removed from `sklearn.utils.deprecate` ensuring that the appropriate depracation message appears in all the docstrings using the `@deprecated` decorator at the moment.


### Suggest a potential alternative/fix

_No response_

COMMENT:
Thanks @maikia for opening this.

Labels: Documentation, Build / CI
log_loss giving nan when input is np.float32 and eps is default

### Describe the bug

When input has values that are numpy array of np.float32, 1-eps (with default eps=1e-15) results in 1.0, and log_loss() when calculating log(1-p) with p=1.0 results in nan.

### Steps/Code to Reproduce

```py
from sklearn.metrics import log_loss
import numpy as np
input = np.array([1],dtype=np.float32)

# when the input is array of np.float32, using the proper eps=eps=np.finfo(np.float32).eps, log_loss is fine
result = log_loss([[0,1]],input,eps=np.finfo(np.float32).eps)
print('with eps=np.finfo(np.float32).eps:',result)

# with input cast as np.float64, log_loss is also fine with the default eps=1e-15
result = log_loss([[0,1]],input.astype(np.float64))
print('with input as np.float64:',result)

# However, the following input as array of np.float32 using the default eps=1e-15 will give nan
result = log_loss([[0,1]],input)
print('with eps=1e-15 (default):',result)
```

### Expected Results

not nan

### Actual Results

with eps=1e-15 (default): nan

```
/Users/gso/anaconda3/lib/python3.9/site-packages/sklearn/metrics/_classification.py:2442: RuntimeWarning: divide by zero encountered in log
  loss = -(transformed_labels * np.log(y_pred)).sum(axis=1)
/Users/gso/anaconda3/lib/python3.9/site-packages/sklearn/metrics/_classification.py:2442: RuntimeWarning: invalid value encountered in multiply
  loss = -(transformed_labels * np.log(y_pred)).sum(axis=1)
```

### Versions

```shell
System:
    python: 3.9.12 (main, Apr  5 2022, 01:53:17)  [Clang 12.0.0 ]
executable: /Users/gso/anaconda3/bin/python
   machine: macOS-10.16-x86_64-i386-64bit

Python dependencies:
          pip: 21.2.4
   setuptools: 61.2.0
      sklearn: 1.0.2
        numpy: 1.21.5
        scipy: 1.7.3
       Cython: 0.29.28
       pandas: 1.4.2
   matplotlib: 3.5.1
       joblib: 1.1.0
threadpoolctl: 2.2.0

Built with OpenMP: True
```

COMMENT:
I traced the behavior from sklearn.metrics.log_loss to np.clip not giving the correct answer for 1-eps, where if the input is numpy array of np.float32, with default eps=1e-15, 1-eps = 1.0, which caused log(1-p) with p=1.0 gives the nan.  I initially went to report this to numpy but they say scikit is not supporting array of np.float32 properly.  Please see:
https://github.com/numpy/numpy/issues/22192

Labels: Bug, help wanted