Talip7/scikit-learn-issues-multilabel huggingface dataset:
RandomForest not passing feature names to trees and creating warnings.

### Describe the bug

I fit a decision forest with training data that includes feature names. When I call predict_proba on the forest everything is fine. When I call rf.estimators_[0].predict_proba it will warn that it was not trained with feature names.


### Steps/Code to Reproduce
```python
import numpy as np
from sklearn.ensemble import RandomForestClassifier

print(sklearn.__version__)

data = np.random.normal([1,2,3,4,5,6,7,8,9,10], size=(1000, 10))
feature_names = [f'F{i}' for i in range(10)]
df = pd.DataFrame(data=data, columns=feature_names)
y = np.ones(1000)
y[500:] = 0

rf = RandomForestClassifier()
rf.fit(df, y)

print(f"Feature names in Forest: {rf.feature_names_in_} ")

print('Classing pred proba on forest')
rf.predict_proba(df)
print('Done calling pred proba on forest')

# THIS GIVES A WARNING
rf.estimators_[0].predict_proba(df)
```

### Expected Results

No warnings

### Actual Results
```
Feature names in Forest: ['F0' 'F1' 'F2' 'F3' 'F4' 'F5' 'F6' 'F7' 'F8' 'F9'] 
Classing pred proba on forest
Done calling pred proba on forest
.....lib/python3.8/site-packages/sklearn/base.py:432: UserWarning: X has feature names, but DecisionTreeClassifier was fitted without feature names
  warnings.warn(
```
### Versions

```shell
sk>>> sklearn.show_versions()

System:
    python: 3.8.16 (default, Mar  2 2023, 03:21:46)  [GCC 11.2.0]
executable: ....../bin/python
   machine: Linux-5.15.0-60-generic-x86_64-with-glibc2.17

Python dependencies:
      sklearn: 1.2.2
          pip: 23.0.1
   setuptools: 65.6.3
        numpy: 1.23.1
        scipy: 1.9.0
       Cython: None
       pandas: 1.5.3
   matplotlib: None
       joblib: 1.1.1
threadpoolctl: 2.2.0
```

COMMENT:
Ok, I've marked this as a good first issue/help wanted as well. If someone else has time before @Howarth, please post a message here so we can avoid duplicate work.

Labels: Bug, Moderate, module:ensemble
RandomForest not passing feature names to trees and creating warnings.

### Describe the bug

I fit a decision forest with training data that includes feature names. When I call predict_proba on the forest everything is fine. When I call rf.estimators_[0].predict_proba it will warn that it was not trained with feature names.


### Steps/Code to Reproduce
```python
import numpy as np
from sklearn.ensemble import RandomForestClassifier

print(sklearn.__version__)

data = np.random.normal([1,2,3,4,5,6,7,8,9,10], size=(1000, 10))
feature_names = [f'F{i}' for i in range(10)]
df = pd.DataFrame(data=data, columns=feature_names)
y = np.ones(1000)
y[500:] = 0

rf = RandomForestClassifier()
rf.fit(df, y)

print(f"Feature names in Forest: {rf.feature_names_in_} ")

print('Classing pred proba on forest')
rf.predict_proba(df)
print('Done calling pred proba on forest')

# THIS GIVES A WARNING
rf.estimators_[0].predict_proba(df)
```

### Expected Results

No warnings

### Actual Results
```
Feature names in Forest: ['F0' 'F1' 'F2' 'F3' 'F4' 'F5' 'F6' 'F7' 'F8' 'F9'] 
Classing pred proba on forest
Done calling pred proba on forest
.....lib/python3.8/site-packages/sklearn/base.py:432: UserWarning: X has feature names, but DecisionTreeClassifier was fitted without feature names
  warnings.warn(
```
### Versions

```shell
sk>>> sklearn.show_versions()

System:
    python: 3.8.16 (default, Mar  2 2023, 03:21:46)  [GCC 11.2.0]
executable: ....../bin/python
   machine: Linux-5.15.0-60-generic-x86_64-with-glibc2.17

Python dependencies:
      sklearn: 1.2.2
          pip: 23.0.1
   setuptools: 65.6.3
        numpy: 1.23.1
        scipy: 1.9.0
       Cython: None
       pandas: 1.5.3
   matplotlib: None
       joblib: 1.1.1
threadpoolctl: 2.2.0
```

COMMENT:
I don't think we want to have the feature names in all trees, as @thomasjpfan mentions, I think the fix here would be to disable validation for sub-trees instead.

Labels: Bug, Moderate, module:ensemble
RandomForest not passing feature names to trees and creating warnings.

### Describe the bug

I fit a decision forest with training data that includes feature names. When I call predict_proba on the forest everything is fine. When I call rf.estimators_[0].predict_proba it will warn that it was not trained with feature names.


### Steps/Code to Reproduce
```python
import numpy as np
from sklearn.ensemble import RandomForestClassifier

print(sklearn.__version__)

data = np.random.normal([1,2,3,4,5,6,7,8,9,10], size=(1000, 10))
feature_names = [f'F{i}' for i in range(10)]
df = pd.DataFrame(data=data, columns=feature_names)
y = np.ones(1000)
y[500:] = 0

rf = RandomForestClassifier()
rf.fit(df, y)

print(f"Feature names in Forest: {rf.feature_names_in_} ")

print('Classing pred proba on forest')
rf.predict_proba(df)
print('Done calling pred proba on forest')

# THIS GIVES A WARNING
rf.estimators_[0].predict_proba(df)
```

### Expected Results

No warnings

### Actual Results
```
Feature names in Forest: ['F0' 'F1' 'F2' 'F3' 'F4' 'F5' 'F6' 'F7' 'F8' 'F9'] 
Classing pred proba on forest
Done calling pred proba on forest
.....lib/python3.8/site-packages/sklearn/base.py:432: UserWarning: X has feature names, but DecisionTreeClassifier was fitted without feature names
  warnings.warn(
```
### Versions

```shell
sk>>> sklearn.show_versions()

System:
    python: 3.8.16 (default, Mar  2 2023, 03:21:46)  [GCC 11.2.0]
executable: ....../bin/python
   machine: Linux-5.15.0-60-generic-x86_64-with-glibc2.17

Python dependencies:
      sklearn: 1.2.2
          pip: 23.0.1
   setuptools: 65.6.3
        numpy: 1.23.1
        scipy: 1.9.0
       Cython: None
       pandas: 1.5.3
   matplotlib: None
       joblib: 1.1.1
threadpoolctl: 2.2.0
```

COMMENT:
> I don't think we want to have the feature names in all trees, as @thomasjpfan mentions, I think the fix here would be to disable validation for sub-trees instead.

This is the current behavior. The random forest disables validation when `predict*` runs. The opening issue is what to do if a DataFrame is passed into a sub-tree directly:

```python
# Trained on dataframe
rf.fit(df, y)

# Raises a warning
rf.estimators_[0].predict_proba(df)
```

If the random forest was trained on a DataFrame, then I think it is reasonable to expect the above to work without warning. This means having the feature names in all trees.

Labels: Bug, Moderate, module:ensemble
RandomForest not passing feature names to trees and creating warnings.

### Describe the bug

I fit a decision forest with training data that includes feature names. When I call predict_proba on the forest everything is fine. When I call rf.estimators_[0].predict_proba it will warn that it was not trained with feature names.


### Steps/Code to Reproduce
```python
import numpy as np
from sklearn.ensemble import RandomForestClassifier

print(sklearn.__version__)

data = np.random.normal([1,2,3,4,5,6,7,8,9,10], size=(1000, 10))
feature_names = [f'F{i}' for i in range(10)]
df = pd.DataFrame(data=data, columns=feature_names)
y = np.ones(1000)
y[500:] = 0

rf = RandomForestClassifier()
rf.fit(df, y)

print(f"Feature names in Forest: {rf.feature_names_in_} ")

print('Classing pred proba on forest')
rf.predict_proba(df)
print('Done calling pred proba on forest')

# THIS GIVES A WARNING
rf.estimators_[0].predict_proba(df)
```

### Expected Results

No warnings

### Actual Results
```
Feature names in Forest: ['F0' 'F1' 'F2' 'F3' 'F4' 'F5' 'F6' 'F7' 'F8' 'F9'] 
Classing pred proba on forest
Done calling pred proba on forest
.....lib/python3.8/site-packages/sklearn/base.py:432: UserWarning: X has feature names, but DecisionTreeClassifier was fitted without feature names
  warnings.warn(
```
### Versions

```shell
sk>>> sklearn.show_versions()

System:
    python: 3.8.16 (default, Mar  2 2023, 03:21:46)  [GCC 11.2.0]
executable: ....../bin/python
   machine: Linux-5.15.0-60-generic-x86_64-with-glibc2.17

Python dependencies:
      sklearn: 1.2.2
          pip: 23.0.1
   setuptools: 65.6.3
        numpy: 1.23.1
        scipy: 1.9.0
       Cython: None
       pandas: 1.5.3
   matplotlib: None
       joblib: 1.1.1
threadpoolctl: 2.2.0
```

COMMENT:
An issue I see with this approach is that it's kind of a hack. Basically, what we're doing here is to apply the mutations which `_validate_data` would have done to the estimator, after the fact, w/o calling `_validate_data`, or having passed data which is different from the data given by the user (like converted to an ndarray version of the data). And the above _fix_ will be immediately out of date if `_validate_data` adds more mutations and we end up doing more checks somehow.

So I see two _clean_-ish solutions here:

- Disable input validation on the estimator level, for sub-estimators of `RandomForest`, so that `rf.estimators_[0].predict(df)` doesn't call validation and doesn't raise/warn anything.
- Or, have a protocol where `_validate_data` tells us what mutations it's doing, and apply the same thing on sub-estimators.

Labels: Bug, Moderate, module:ensemble
RandomForest not passing feature names to trees and creating warnings.

### Describe the bug

I fit a decision forest with training data that includes feature names. When I call predict_proba on the forest everything is fine. When I call rf.estimators_[0].predict_proba it will warn that it was not trained with feature names.


### Steps/Code to Reproduce
```python
import numpy as np
from sklearn.ensemble import RandomForestClassifier

print(sklearn.__version__)

data = np.random.normal([1,2,3,4,5,6,7,8,9,10], size=(1000, 10))
feature_names = [f'F{i}' for i in range(10)]
df = pd.DataFrame(data=data, columns=feature_names)
y = np.ones(1000)
y[500:] = 0

rf = RandomForestClassifier()
rf.fit(df, y)

print(f"Feature names in Forest: {rf.feature_names_in_} ")

print('Classing pred proba on forest')
rf.predict_proba(df)
print('Done calling pred proba on forest')

# THIS GIVES A WARNING
rf.estimators_[0].predict_proba(df)
```

### Expected Results

No warnings

### Actual Results
```
Feature names in Forest: ['F0' 'F1' 'F2' 'F3' 'F4' 'F5' 'F6' 'F7' 'F8' 'F9'] 
Classing pred proba on forest
Done calling pred proba on forest
.....lib/python3.8/site-packages/sklearn/base.py:432: UserWarning: X has feature names, but DecisionTreeClassifier was fitted without feature names
  warnings.warn(
```
### Versions

```shell
sk>>> sklearn.show_versions()

System:
    python: 3.8.16 (default, Mar  2 2023, 03:21:46)  [GCC 11.2.0]
executable: ....../bin/python
   machine: Linux-5.15.0-60-generic-x86_64-with-glibc2.17

Python dependencies:
      sklearn: 1.2.2
          pip: 23.0.1
   setuptools: 65.6.3
        numpy: 1.23.1
        scipy: 1.9.0
       Cython: None
       pandas: 1.5.3
   matplotlib: None
       joblib: 1.1.1
threadpoolctl: 2.2.0
```

COMMENT:
Somehow the cleanest solution would be to go for a "dataframes all the way down" approach, where things are only converted as late as necessary. Compared to the current setup where conversion happens very early. This is also what https://github.com/scikit-learn/scikit-learn/pull/21811#issuecomment-1091869844 proposes as a way forward. The idea of splitting `check_array` so that it does only checking, not checking and converting has come up a few times as well. There is also #25433 (and friends) that would also like a move in that direction.

What was the thinking behind the current "early conversion" approach?

Labels: Bug, Moderate, module:ensemble
RandomForest not passing feature names to trees and creating warnings.

### Describe the bug

I fit a decision forest with training data that includes feature names. When I call predict_proba on the forest everything is fine. When I call rf.estimators_[0].predict_proba it will warn that it was not trained with feature names.


### Steps/Code to Reproduce
```python
import numpy as np
from sklearn.ensemble import RandomForestClassifier

print(sklearn.__version__)

data = np.random.normal([1,2,3,4,5,6,7,8,9,10], size=(1000, 10))
feature_names = [f'F{i}' for i in range(10)]
df = pd.DataFrame(data=data, columns=feature_names)
y = np.ones(1000)
y[500:] = 0

rf = RandomForestClassifier()
rf.fit(df, y)

print(f"Feature names in Forest: {rf.feature_names_in_} ")

print('Classing pred proba on forest')
rf.predict_proba(df)
print('Done calling pred proba on forest')

# THIS GIVES A WARNING
rf.estimators_[0].predict_proba(df)
```

### Expected Results

No warnings

### Actual Results
```
Feature names in Forest: ['F0' 'F1' 'F2' 'F3' 'F4' 'F5' 'F6' 'F7' 'F8' 'F9'] 
Classing pred proba on forest
Done calling pred proba on forest
.....lib/python3.8/site-packages/sklearn/base.py:432: UserWarning: X has feature names, but DecisionTreeClassifier was fitted without feature names
  warnings.warn(
```
### Versions

```shell
sk>>> sklearn.show_versions()

System:
    python: 3.8.16 (default, Mar  2 2023, 03:21:46)  [GCC 11.2.0]
executable: ....../bin/python
   machine: Linux-5.15.0-60-generic-x86_64-with-glibc2.17

Python dependencies:
      sklearn: 1.2.2
          pip: 23.0.1
   setuptools: 65.6.3
        numpy: 1.23.1
        scipy: 1.9.0
       Cython: None
       pandas: 1.5.3
   matplotlib: None
       joblib: 1.1.1
threadpoolctl: 2.2.0
```

COMMENT:
> What was the thinking behind the current "early conversion" approach?

1. Performance: Validation and multiple conversion can be VERY expensive. Worst case we'd have a whole data copy when converting to ndarray every time. That's also why we have things like this in the meta-estimator level:
```py
        if issparse(X):
            # Pre-sort indices to avoid that each individual tree of the
            # ensemble sorts the indices.
            X.sort_indices()
```
2. In some cases we need to know certain information about the input already in the meta-estimator level, which can be avoided in the `BaseForest` case it seems, but the performance reason stands. We do bootstrapping there, but we can do it w/o conversion, which probably would be more expensive too.

So we can only _delay_ validation when:
1. we don't need info / modifications on the input
2. we only fit one sub-estimator, so we won't have the issue of multiple validations

Labels: Bug, Moderate, module:ensemble
RandomForest not passing feature names to trees and creating warnings.

### Describe the bug

I fit a decision forest with training data that includes feature names. When I call predict_proba on the forest everything is fine. When I call rf.estimators_[0].predict_proba it will warn that it was not trained with feature names.


### Steps/Code to Reproduce
```python
import numpy as np
from sklearn.ensemble import RandomForestClassifier

print(sklearn.__version__)

data = np.random.normal([1,2,3,4,5,6,7,8,9,10], size=(1000, 10))
feature_names = [f'F{i}' for i in range(10)]
df = pd.DataFrame(data=data, columns=feature_names)
y = np.ones(1000)
y[500:] = 0

rf = RandomForestClassifier()
rf.fit(df, y)

print(f"Feature names in Forest: {rf.feature_names_in_} ")

print('Classing pred proba on forest')
rf.predict_proba(df)
print('Done calling pred proba on forest')

# THIS GIVES A WARNING
rf.estimators_[0].predict_proba(df)
```

### Expected Results

No warnings

### Actual Results
```
Feature names in Forest: ['F0' 'F1' 'F2' 'F3' 'F4' 'F5' 'F6' 'F7' 'F8' 'F9'] 
Classing pred proba on forest
Done calling pred proba on forest
.....lib/python3.8/site-packages/sklearn/base.py:432: UserWarning: X has feature names, but DecisionTreeClassifier was fitted without feature names
  warnings.warn(
```
### Versions

```shell
sk>>> sklearn.show_versions()

System:
    python: 3.8.16 (default, Mar  2 2023, 03:21:46)  [GCC 11.2.0]
executable: ....../bin/python
   machine: Linux-5.15.0-60-generic-x86_64-with-glibc2.17

Python dependencies:
      sklearn: 1.2.2
          pip: 23.0.1
   setuptools: 65.6.3
        numpy: 1.23.1
        scipy: 1.9.0
       Cython: None
       pandas: 1.5.3
   matplotlib: None
       joblib: 1.1.1
threadpoolctl: 2.2.0
```

COMMENT:
I could get behind the idea of turning off input validation for the sub-estimators or adding the information needed after calling fit from the parent estimator (the "hack").

One thing I just thought about: what happens if the order of the columns changes between the `df` used for `fit` and `predict`? And how does this effect the current topic of passing a `df` to a sub-estimator that was fitted on a numpy array?

Labels: Bug, Moderate, module:ensemble
RandomForest not passing feature names to trees and creating warnings.

### Describe the bug

I fit a decision forest with training data that includes feature names. When I call predict_proba on the forest everything is fine. When I call rf.estimators_[0].predict_proba it will warn that it was not trained with feature names.


### Steps/Code to Reproduce
```python
import numpy as np
from sklearn.ensemble import RandomForestClassifier

print(sklearn.__version__)

data = np.random.normal([1,2,3,4,5,6,7,8,9,10], size=(1000, 10))
feature_names = [f'F{i}' for i in range(10)]
df = pd.DataFrame(data=data, columns=feature_names)
y = np.ones(1000)
y[500:] = 0

rf = RandomForestClassifier()
rf.fit(df, y)

print(f"Feature names in Forest: {rf.feature_names_in_} ")

print('Classing pred proba on forest')
rf.predict_proba(df)
print('Done calling pred proba on forest')

# THIS GIVES A WARNING
rf.estimators_[0].predict_proba(df)
```

### Expected Results

No warnings

### Actual Results
```
Feature names in Forest: ['F0' 'F1' 'F2' 'F3' 'F4' 'F5' 'F6' 'F7' 'F8' 'F9'] 
Classing pred proba on forest
Done calling pred proba on forest
.....lib/python3.8/site-packages/sklearn/base.py:432: UserWarning: X has feature names, but DecisionTreeClassifier was fitted without feature names
  warnings.warn(
```
### Versions

```shell
sk>>> sklearn.show_versions()

System:
    python: 3.8.16 (default, Mar  2 2023, 03:21:46)  [GCC 11.2.0]
executable: ....../bin/python
   machine: Linux-5.15.0-60-generic-x86_64-with-glibc2.17

Python dependencies:
      sklearn: 1.2.2
          pip: 23.0.1
   setuptools: 65.6.3
        numpy: 1.23.1
        scipy: 1.9.0
       Cython: None
       pandas: 1.5.3
   matplotlib: None
       joblib: 1.1.1
threadpoolctl: 2.2.0
```

COMMENT:
Hello, I'm a first timer and new to open source contribution,
Can I take up this issue ?

Labels: Bug, Moderate, module:ensemble
RandomForest not passing feature names to trees and creating warnings.

### Describe the bug

I fit a decision forest with training data that includes feature names. When I call predict_proba on the forest everything is fine. When I call rf.estimators_[0].predict_proba it will warn that it was not trained with feature names.


### Steps/Code to Reproduce
```python
import numpy as np
from sklearn.ensemble import RandomForestClassifier

print(sklearn.__version__)

data = np.random.normal([1,2,3,4,5,6,7,8,9,10], size=(1000, 10))
feature_names = [f'F{i}' for i in range(10)]
df = pd.DataFrame(data=data, columns=feature_names)
y = np.ones(1000)
y[500:] = 0

rf = RandomForestClassifier()
rf.fit(df, y)

print(f"Feature names in Forest: {rf.feature_names_in_} ")

print('Classing pred proba on forest')
rf.predict_proba(df)
print('Done calling pred proba on forest')

# THIS GIVES A WARNING
rf.estimators_[0].predict_proba(df)
```

### Expected Results

No warnings

### Actual Results
```
Feature names in Forest: ['F0' 'F1' 'F2' 'F3' 'F4' 'F5' 'F6' 'F7' 'F8' 'F9'] 
Classing pred proba on forest
Done calling pred proba on forest
.....lib/python3.8/site-packages/sklearn/base.py:432: UserWarning: X has feature names, but DecisionTreeClassifier was fitted without feature names
  warnings.warn(
```
### Versions

```shell
sk>>> sklearn.show_versions()

System:
    python: 3.8.16 (default, Mar  2 2023, 03:21:46)  [GCC 11.2.0]
executable: ....../bin/python
   machine: Linux-5.15.0-60-generic-x86_64-with-glibc2.17

Python dependencies:
      sklearn: 1.2.2
          pip: 23.0.1
   setuptools: 65.6.3
        numpy: 1.23.1
        scipy: 1.9.0
       Cython: None
       pandas: 1.5.3
   matplotlib: None
       joblib: 1.1.1
threadpoolctl: 2.2.0
```

COMMENT:
Hi, is this available to be taken?

Labels: Bug, Moderate, module:ensemble
RandomForest not passing feature names to trees and creating warnings.

### Describe the bug

I fit a decision forest with training data that includes feature names. When I call predict_proba on the forest everything is fine. When I call rf.estimators_[0].predict_proba it will warn that it was not trained with feature names.


### Steps/Code to Reproduce
```python
import numpy as np
from sklearn.ensemble import RandomForestClassifier

print(sklearn.__version__)

data = np.random.normal([1,2,3,4,5,6,7,8,9,10], size=(1000, 10))
feature_names = [f'F{i}' for i in range(10)]
df = pd.DataFrame(data=data, columns=feature_names)
y = np.ones(1000)
y[500:] = 0

rf = RandomForestClassifier()
rf.fit(df, y)

print(f"Feature names in Forest: {rf.feature_names_in_} ")

print('Classing pred proba on forest')
rf.predict_proba(df)
print('Done calling pred proba on forest')

# THIS GIVES A WARNING
rf.estimators_[0].predict_proba(df)
```

### Expected Results

No warnings

### Actual Results
```
Feature names in Forest: ['F0' 'F1' 'F2' 'F3' 'F4' 'F5' 'F6' 'F7' 'F8' 'F9'] 
Classing pred proba on forest
Done calling pred proba on forest
.....lib/python3.8/site-packages/sklearn/base.py:432: UserWarning: X has feature names, but DecisionTreeClassifier was fitted without feature names
  warnings.warn(
```
### Versions

```shell
sk>>> sklearn.show_versions()

System:
    python: 3.8.16 (default, Mar  2 2023, 03:21:46)  [GCC 11.2.0]
executable: ....../bin/python
   machine: Linux-5.15.0-60-generic-x86_64-with-glibc2.17

Python dependencies:
      sklearn: 1.2.2
          pip: 23.0.1
   setuptools: 65.6.3
        numpy: 1.23.1
        scipy: 1.9.0
       Cython: None
       pandas: 1.5.3
   matplotlib: None
       joblib: 1.1.1
threadpoolctl: 2.2.0
```

COMMENT:
There is already a PR opened: https://github.com/scikit-learn/scikit-learn/pull/26208, but we do not have an agreed solution.

Labels: Bug, Moderate, module:ensemble
HistGradientBoosting counts and sample weights

Related issues: #25210

### Current State
`HistGradientBootingClassifier` and `HistGradientBootingRegressor` both:
- Calculate the sample size `count` in histograms
- Use `count` for splitting (mostly excluding split candidates)
- Save the `count` in the final trees and use it in partial dependence computations.

### Proposition
1. Evaluate if removing `count` from the histograms (LightGBM only sums gradient and hessian in histograms, no count) gives a good speed-up .(I measured a roughly 10-20% speed-up.)
  Edit: LightGBM uses an approximate count based on the hessian to check for min sample size. So this might not be what we want.
3. Add an option to save counts and sample weights to final trees at the very end of `fit` (where the binned training `X` is still available).
4. Use partial dependence `method='recursion'` if the above option was set, else use `method='brute'`.

Why?
#25431 concluded that adding weights to the trees is too expensive. The above proposition gives a user a clear choice: Faster training time or faster pdp afterwards.

COMMENT:
Hello, I'm a first timer and new to open source,
Can I take up this issue ?

Labels: New Feature, Performance, Needs Decision, Needs Benchmarks, module:ensemble, Breaking Change