Discrepancy between output of classifier feature_importances_ with different sklearn installations

### Describe the bug

I am currently using `scikit-learn` classifier `feature_importances_` attribute on a project to rank important features from my model, and my `CI` pipeline runs the project test-suite using instances of `scikit-learn==1.3.2` and `scikit-learn==1.5.2` on a remote linux host. I am experiencing some discrepancies in the output of the relevant test (for which I have provided a minimal viable reproducer below) on different machines/installations/sklearn versions. 

There are a few specific problems I am experiencing:

1. Locally, the test will pass using a binary installation of `scikit-learn==1.3.2` and fail using `scikit-learn==1.5.2`. With the help of my team, we have traced this error back and found the earliest failing version to be `1.4.1.post1`.  We suspect that the error originates from a change made in https://github.com/scikit-learn/scikit-learn/pull/27639 that has to do with the switch from absolute counts to store proportions in `tree_.values` but have not determined a root cause for the discrepancy.
2. As mentioned in (1) when running the test-suite locally on my `Mac-ARM64` machine, the test will fail as described, however, when running the test on a remote linux machine, the test will pass with both sklearn versions
3. The test will fail when I build the code from source vs. from the binary distribution of `scikit-learn==1.3.2`

My main question is, what could be the cause of these observed discrepancies between sklearn version, installation type and environment and which output is most "correct"?

### Steps/Code to Reproduce

```python
from sklearn.ensemble import RandomForestClassifier
import pandas as pd
import numpy as np
from pandas.testing import assert_frame_equal
import pdb


# this test serves as a minimal viable reproducer for the
# difference observed in output of tree values between
# sklearn versions 1.3.2 and 1.4.2. this test should pass
# when using sklearn==1.3.2 and fail when using sklearn==1.4.2

# first create a minimal dataset of values for training
# a Random Forest Classifier
random_state = 123
rng = np.random.default_rng(random_state)
X = rng.integers(0, 2, size=(1000, 12))
y = np.asarray([1, 0] * 500)
y[-1] = 1

X[:, 0] = y
X[:, 1] = 1 - y
clfr = RandomForestClassifier(n_estimators=100, random_state=random_state)
clfr.fit(X, y)

# find the importances of the estimator and check that ranking of the importances
importances_out = clfr.feature_importances_
# these are the importance values that are expected from ``sklearn==1.3.2``
importances_exp = np.array(
    [
        0.52090464,
        0.46263368,
        0.00115268,
        0.00179985,
        0.00177495,
        0.00169134,
        0.00157653,
        0.00135364,
        0.00175814,
        0.00169148,
        0.00162767,
        0.00203539,
    ]
)
importances_out = pd.DataFrame(importances_out).sort_values(by=0)
importances_exp = pd.DataFrame(importances_exp).sort_values(by=0)

assert_frame_equal(importances_out, importances_exp)
```


### Expected Results

The expected values above, i.e. `importances_exp` represent the ranked feature importances of a "cooked" dataset where the input to the model is an array of random values, except two rows, which are perfectly (inversely) correlated to the target values `y`.  As we expect, the two highly correlated values show the highest importance and the random values show the lowest importance. The test checks that the ranking of the input values is correct by comparing the `DataFrames` storing the sorted output values from `clfr.feature_importances_`.

### Actual Results

The expected output above, which comes from `feature_importances_` when using `scikit-learn==1.3.2` differs by some floating point values from the output  when using `scikit-learn>=1.4.2`, i.e.:

```python
array([[0.52087185],
       [0.46270245],
       [0.00203539],
       [0.00179985],
       [0.00178587],
       [0.00177495],
       [0.00169148],
       [0.00169134],
       [0.00159994],
       [0.00155658],
       [0.00135364],
       [0.00113665]])
```

where the ranking of the values is changed by the discrepancy between floating point values of the lower ranked features:

```bash
DataFrame.index values are different (16.66667 %)
[left]:  Index([2, 7, 6, 10, 5, 9, 4, 8, 3, 11, 1, 0], dtype='int64')
[right]: Index([2, 7, 6, 10, 5, 9, 8, 4, 3, 11, 1, 0], dtype='int64')
At positional index 6, first diff: 4 != 8
```

### Versions

```shell
On `Mac-arm64` with `scikit-learn==1.3.2`:

System:
    python: 3.11.7 (main, Dec  4 2023, 18:10:11) [Clang 15.0.0 (clang-1500.1.0.2.5)]
executable: /Users/awitmer/venv_7/bin/python
   machine: macOS-14.7.5-arm64-arm-64bit

Python dependencies:
      sklearn: 1.3.2
          pip: 23.3.1
   setuptools: 69.0.2
        numpy: 1.25.0
        scipy: 1.14.0
       Cython: None
       pandas: 2.0.2
   matplotlib: 3.7.1
       joblib: 1.2.0
threadpoolctl: 3.1.0

Built with OpenMP: True

threadpoolctl info:
       user_api: openmp
   internal_api: openmp
         prefix: libomp
       filepath: /Users/awitmer/venv_7/lib/python3.11/site-packages/sklearn/.dylibs/libomp.dylib
        version: None
    num_threads: 12

       user_api: blas
   internal_api: openblas
         prefix: libopenblas
       filepath: /Users/awitmer/venv_7/lib/python3.11/site-packages/numpy/.dylibs/libopenblas64_.0.dylib
        version: 0.3.23
threading_layer: pthreads
   architecture: armv8
    num_threads: 12

On `Mac-arm64` with `scikit-learn==1.5.2`:

System:
    python: 3.11.7 (main, Dec  4 2023, 18:10:11) [Clang 15.0.0 (clang-1500.1.0.2.5)]
executable: /Users/awitmer/venv_7/bin/python
   machine: macOS-14.7.5-arm64-arm-64bit

Python dependencies:
      sklearn: 1.5.2
          pip: 23.3.1
   setuptools: 69.0.2
        numpy: 1.25.0
        scipy: 1.14.0
       Cython: None
       pandas: 2.0.2
   matplotlib: 3.7.1
       joblib: 1.2.0
threadpoolctl: 3.1.0

Built with OpenMP: True

threadpoolctl info:
       user_api: blas
   internal_api: openblas
         prefix: libopenblas
       filepath: /Users/awitmer/venv_7/lib/python3.11/site-packages/numpy/.dylibs/libopenblas64_.0.dylib
        version: 0.3.23
threading_layer: pthreads
   architecture: armv8
    num_threads: 12

       user_api: openmp
   internal_api: openmp
         prefix: libomp
       filepath: /Users/awitmer/venv_7/lib/python3.11/site-packages/sklearn/.dylibs/libomp.dylib
        version: None
    num_threads: 12

On linux based server with `scikit-learn==1.3.2`:

System:
    python: 3.11.11 (main, Dec  9 2024, 15:32:27) [GCC 8.5.0 20210514 (Red Hat 8.5.0-22)]
executable: /usr/bin/python3.11
   machine: Linux-4.18.0-553.47.1.el8_10.x86_64-x86_64-with-glibc2.28

Python dependencies:
      sklearn: 1.3.2
          pip: 25.0.1
   setuptools: 65.5.1
        numpy: 1.26.4
        scipy: 1.15.1
       Cython: None
       pandas: 2.2.3
   matplotlib: 3.9.0
       joblib: 1.2.0
threadpoolctl: 3.1.0

Built with OpenMP: True

threadpoolctl info:
       user_api: openmp
   internal_api: openmp
         prefix: libgomp
       filepath: /vast/home/awitmer/.local/lib/python3.11/site-packages/scikit_learn.libs/libgomp-a34b3233.so.1.0.0
        version: None
    num_threads: 40

       user_api: blas
   internal_api: openblas
         prefix: libopenblas
       filepath: /vast/home/awitmer/.local/lib/python3.11/site-packages/numpy.libs/libopenblas64_p-r0-0cf96a72.3.23.dev.so
        version: 0.3.23.dev
threading_layer: pthreads
   architecture: Haswell
    num_threads: 40
```

COMMENT:
We certainly don't support comparing such values across versions.

And if you look at it, you can consider the disparity coming from floating point disparities. The actual difference in the feature importance values are minute.

As for which one is "more correct", we hope the latest release of course. That's why things change, people report issues, or we find them, and we fix them. That very often results in getting different results if you compare versions.

Labels: Bug, Needs Triage