Repository: scikit-learn/scikit-learn

Issue:
# Gradient Boosting: Tree splitting criterion`"friedman_mse"` isn't different from normal `"mse"`

### Describe the bug

In `sklearn/tree/_criterion.pyx`, criteria `"friedman_mse"` (class `FriedmanMSE(MSE)`) and `"squared_error"` (class `MSE`) are mathematically equivalent, they use different but equivalent formulas. This is confusing, and I think that:
- `"friedman_mse"` criterion should be deprecated then removed.
- the class should be removed

Because `"friedman_mse"` and `"squared_error"` are mathematically equivalent they produce the same trees/forests (up to float precision errors), as shown by the reproducer below.

Also, I tried removing all the code from `FriedmanMSE` class (leaving only the inheritance `FriedmanMSE(MSE)`) and all test passes from `sklearn/tree` and `sklearn/ensemble` except `test_huber_exact_backward_compat` which is sensitive to float rounding errors.
I also tried the other-way around (copy-pasting code from `FriedmanMSE(MSE)` in `MSE`) and except for one bug for which I'll open a separate issue, it worked as well.


Maths (from ChatGPT, but I carefully double-checked it):

<img width="1151" height="565" alt="image" src="https://github.com/user-attachments/assets/7523b114-9c7b-4a20-abeb-dcb2c6d158fc" />

Proof:

<details>

<img width="1271" height="857" alt="image" src="https://github.com/user-attachments/assets/83a92ec6-0f5e-4387-a9d3-fb1b397b0302" />

</details>



### Steps/Code to Reproduce


```Python
import numpy as np
from sklearn.tree import DecisionTreeRegressor
from sklearn.ensemble import RandomForestRegressor, GradientBoostingRegressor


params_list = [
    {
        'random_state': 0,
        'max_leaf_nodes': max_leaf_nodes,
        'max_depth': max_depth,
        'ccp_alpha': ccp_alpha,
    }
    for max_depth in [None, 1, 2, 3, 5, 10]
    for max_leaf_nodes in [None, 3, 6, 10, 20, 50, 100]
    for ccp_alpha in [0, 0.005, 0.01]
]

for Reg in [DecisionTreeRegressor, RandomForestRegressor, GradientBoostingRegressor]:
    print(Reg)
    for params in params_list:
        n = np.random.choice([10, 100, 1000, 10_000])
        d = np.random.choice([1, 2, 3, 10])
        if Reg in [GradientBoostingRegressor, RandomForestRegressor]:
            params = {**params, 'n_estimators': 10}
        X = np.random.rand(n, 1)
        X_test = np.random.rand(n, 1)
        y = np.random.rand(n) + X.sum(axis=1)

        for w  in [None, np.random.rand(n), np.random.pareto(1, size=n)]:
            tree_mse = Reg(criterion="squared_error", **params)
            tree_friedman = Reg(criterion="friedman_mse",  **params)

            assert np.allclose(
                tree_mse.fit(X, y, sample_weight=w).predict(X_test),
                tree_friedman.fit(X, y, sample_weight=w).predict(X_test)
            )
            if Reg is DecisionTreeRegressor:
                assert np.allclose(tree_mse.tree_.impurity, tree_friedman.tree_.impurity)
```

### Expected Results

You would expect different criteria to give different results and hence this code to raise.

### Actual Results

Doesn't raise.

### Versions

```shell
System:
    python: 3.12.11 (main, Aug 18 2025, 19:19:11) [Clang 20.1.4 ]
executable: /home/arthur/dev-perso/scikit-learn/sklearn-env/bin/python
   machine: Linux-6.14.0-35-generic-x86_64-with-glibc2.39

Python dependencies:
      sklearn: 1.8.dev0
          pip: None
   setuptools: 80.9.0
        numpy: 2.3.4
        scipy: 1.16.2
       Cython: 3.1.5
       pandas: 2.3.3
   matplotlib: 3.10.7
       joblib: 1.5.2
threadpoolctl: 3.6.0

Built with OpenMP: True

threadpoolctl info:
       user_api: blas
   internal_api: openblas
    num_threads: 16
         prefix: libscipy_openblas
       filepath: /home/arthur/dev-perso/scikit-learn/sklearn-env/lib/python3.12/site-packages/numpy.libs/libscipy_openblas64_-8fb3d286.so
        version: 0.3.30
threading_layer: pthreads
   architecture: Haswell

       user_api: blas
   internal_api: openblas
    num_threads: 16
         prefix: libscipy_openblas
       filepath: /home/arthur/dev-perso/scikit-learn/sklearn-env/lib/python3.12/site-packages/scipy.libs/libscipy_openblas-b75cc656.so
        version: 0.3.29.dev
threading_layer: pthreads
   architecture: Haswell

       user_api: openmp
   internal_api: openmp
    num_threads: 16
         prefix: libgomp
       filepath: /usr/lib/x86_64-linux-gnu/libgomp.so.1.0.0
        version: None
```

Implementation patch:
diff --git a/sklearn/ensemble/_forest.py b/sklearn/ensemble/_forest.py
index b7b4707dcaa0e..e1d666a8bf50f 100644
--- a/sklearn/ensemble/_forest.py
+++ b/sklearn/ensemble/_forest.py
@@ -1928,6 +1928,16 @@ def __init__(
             max_samples=max_samples,
         )
 
+        if isinstance(criterion, str) and criterion == "friedman_mse":
+            # TODO(1.11): remove support of "friedman_mse" criterion.
+            criterion = "squared_error"
+            warn(
+                'Value `"friedman_mse"` for `criterion` is deprecated and will be '
+                'removed in 1.11. It maps to `"squared_error"` as both '
+                'were always equivalent. Use `criterion="squared_error"` '
+                "to remove this warning.",
+                FutureWarning,
+            )
         self.criterion = criterion
         self.max_depth = max_depth
         self.min_samples_split = min_samples_split
@@ -2662,6 +2672,16 @@ def __init__(
             max_samples=max_samples,
         )
 
+        if isinstance(criterion, str) and criterion == "friedman_mse":
+            # TODO(1.11): remove support of "friedman_mse" criterion.
+            criterion = "squared_error"
+            warn(
+                'Value `"friedman_mse"` for `criterion` is deprecated and will be '
+                'removed in 1.11. It maps to `"squared_error"` as both '
+                'were always equivalent. Use `criterion="squared_error"` '
+                "to remove this warning.",
+                FutureWarning,
+            )
         self.criterion = criterion
         self.max_depth = max_depth
         self.min_samples_split = min_samples_split


Test patch:
diff --git a/sklearn/ensemble/tests/test_forest.py b/sklearn/ensemble/tests/test_forest.py
index 20a452ecb75c4..b04c46c845d27 100644
--- a/sklearn/ensemble/tests/test_forest.py
+++ b/sklearn/ensemble/tests/test_forest.py
@@ -157,8 +157,12 @@ def test_iris_criterion(name, criterion):
     assert score > 0.5, "Failed with criterion %s and score = %f" % (criterion, score)
 
 
+# TODO(1.11): remove the deprecated friedman_mse criterion parametrization
+@pytest.mark.filterwarnings("ignore:.*friedman_mse.*:FutureWarning")
 @pytest.mark.parametrize("name", FOREST_REGRESSORS)
-@pytest.mark.parametrize("criterion", ("squared_error", "absolute_error"))
+@pytest.mark.parametrize(
+    "criterion", ("squared_error", "friedman_mse", "absolute_error")
+)
 def test_regression_criterion(name, criterion):
     # Check consistency on regression dataset.
     ForestRegressor = FOREST_REGRESSORS[name]
@@ -1864,3 +1868,10 @@ def test_non_supported_criterion_raises_error_with_missing_values(Forest):
     msg = ".*does not accept missing values"
     with pytest.raises(ValueError, match=msg):
         forest.fit(X, y)
+
+
+# TODO(1.11): remove test with the deprecation of friedman_mse criterion
+@pytest.mark.parametrize("Forest", FOREST_REGRESSORS.values())
+def test_friedman_mse_deprecation(Forest):
+    with pytest.warns(FutureWarning, match="friedman_mse"):
+        _ = Forest(criterion="friedman_mse")