RFC Unify old GradientBoosting estimators and HGBT

### Current situation
We have the unfortunate situation to have 2 different versions of gradient boosting, the old estimators ([`GradientBoostingClassifier`](https://scikit-learn.org/stable/modules/generated/sklearn.ensemble.GradientBoostingClassifier.html#sklearn-ensemble-gradientboostingclassifier) and `GradientBoostingRegressor`) as well as the new ones using binning and histogram strategies similar to LightGBM ([`HistGradientBoostingClassifier`](https://scikit-learn.org/stable/modules/generated/sklearn.ensemble.HistGradientBoostingClassifier.html#sklearn-ensemble-histgradientboostingclassifier) and `HistGradientBoostingRegressor`).

This makes advertising the new ones harder, e.g. #26826, and also result in a larger feature gap between those two.
Based on discussions in #27139 and during a monthly meeting (maybe not documented), **I'd like to call for comments on the following:**

#### Proposition
Unify both types of gradient boosting in a single class, i.e. the old names `GradientBoostingClassifier` and make them switch the underlying estimator class based on a parameter value, e.g. `max_bins` (`None`->old classes, integer->new classes).

Note that binning and histograms are not the only difference.

### Comparison
#### Algorithm
The old GBT uses Friedman gradient boosting with a line search step. (The lines search sometimes, e.g. for log loss, uses a 2. order approximation and is therefore, sometimes, called "hybrid gradient-Newton boosting"). The trees are learned on the gradients. A tree searches for the best split among all (veeeery many) split candidates for all features. After a single tree is fit, the terminal node values are re-computed which corresponds to a line search step.

The new HGBT uses a 2. order approximation of the loss, i.e. gradients and hessians (XGBoost paper, therefore sometimes called Newton boosting). In addition, it bins/discretizes the features `X` and uses a histogram of gradients/hessians/counts per feature. A tree then searches for the best split candidate, but there are only n_features * n_bins candidates (muuuuch less than in GBT).

| estimator | trees train on        | node values (consequence of tree train) | features `X`   |
| --------- | --------------------- | ---------------------------- | -------------- |
| GBT       | gradients             | recomputed in lines search   | use as is      |
| HGBT      | gradients/hessians    | `sum(gradient)/sum(hessian)` | bin/discretize |

In fact, one could use 2. order loss (gradients and hessians) without binning `X`, and vice-versa, use binning with fitting trees on gradients (without hessians).

#### Parameters

| `HistGradientBoostingRegressor` | `GradientBoostingRegressor` | Same               | Comment |
| ------------------------------- | ----------------------------| ------------------ | ------- |
| loss                            | loss                        | :white_check_mark: | |
| quantile                        | alpha                       | :x:                | | 
| learning_rate                   | learning_rate               | :white_check_mark: | |
| max_iter                        | n_estimators                | :x:                | https://github.com/scikit-learn/scikit-learn/pull/12807#issue-391902947 |
| max_leaf_nodes                  | max_leaf_nodes              | :white_check_mark: |
| max_depth                       | max_depth                   | :white_check_mark: |
| min_samples_leaf                | min_samples_leaf            | :white_check_mark: |
| l2_regularization               | learning_rate               | :white_check_mark: |
| max_features                    | max_features                | :white_check_mark: |
| max_bins                        | :no_entry: (nonsense)       | :x:                |
| categorical_features            | :no_entry:                  | :x:                |
| monotonic_cst                   | :no_entry:                  | :x:                | #27305 |
| interaction_cst                 | :no_entry:                  | :x:                |
| warm_start                      | warm_start                  | :white_check_mark: |
| early_stopping                  | :no_entry:                  | :x:                |
| scoring                         | :no_entry:                  | :x:                |
| validation_fraction             | validation_fraction         | :white_check_mark: |
| n_iter_no_change                | n_iter_no_change            | :white_check_mark: |
| tol                             | tol                         | :white_check_mark: |
| verbose                         | verbose                     | :white_check_mark: |
| random_state                    | random_state                | :white_check_mark: |
| class_weight                    | :no_entry:                  | :x:                |
| :no_entry:                      | subsample                   | :x:                | #16062 |
| :no_entry: (nonsense)           | criterion                   | :x:                |
| :no_entry:                      | min_samples_split           | :x:                |
| :no_entry:                      | min_weight_fraction_leaf    | :x:                |
| :no_entry:                      | min_impurity_decrease       | :x:                |
| :no_entry:                      | init                        | :x:                | #27109 |
| :no_entry:                      | ccp_alpha                   | :x:                |

In fact, only the quantile/alpha and max_iter/n_estimator parameters are conflicting.

COMMENT:
An idea I had relating to this issue: what do people think of a small tool that transforms user's code? It could either be an automatic tool that makes the required edits or maybe more like a linter that points you to things that you should investigate. Taking it one step further, a linter for your scikit-learn usage would be cool. It could point out transformations you could make, anti-patterns, etc

Labels: RFC