Repository: scikit-learn/scikit-learn
File: examples/ensemble/plot_stack_predictors.py (Python)

Code before:
stacking_regressor.fit(X, y)
stacking_regressor.final_estimator_.coef_

# %%
# How to mimic SuperLearner with scikit-learn
# ###########################################
#
# The `SuperLearner` [Polley2010]_ is a stacking strategy implemented as `an R
# package <https://cran.r-project.org/web/packages/SuperLearner/index.html>`_, but
# not available off-the-shelf in Python. It is closely related to the
# :class:`~sklearn.ensemble.StackingRegressor`, as both train the meta-model on
# out-of-fold predictions from the base estimators.
#
# The key difference is that `SuperLearner` estimates a convex set of
# meta-weights (non-negative and summing to 1) and omits an intercept; by
# contrast, :class:`~sklearn.ensemble.StackingRegressor` uses an unconstrained
# meta-learner with an intercept by default (and can optionally include raw
# features via passthrough).
#
# Without an intercept, the meta-weights are directly interpretable as
# fractional contributions to the final prediction.

from sklearn.linear_model import LinearRegression

linear_reg = LinearRegression(fit_intercept=False, positive=True)
stacking_regressor = StackingRegressor(
    estimators=estimators, final_estimator=linear_reg
)
stacking_regressor.fit(X, y)
stacking_regressor.final_estimator_.coef_

# %%
# The sum of meta-weights in the stacked regressor is close to 1.0, but not
# exactly one:

stacking_regressor.final_estimator_.coef_.sum()

# %%
# Beyond interpretability, the normalization to 1.0 constraint in the `SuperLearner`
# presents the following advantages:
#
# - Consensus-preserving: if all base models output the same value at a point,
#   the ensemble returns that same value (no artificial amplification or
#   attenuation).
# - Translation-equivariant: adding a constant to every base prediction shifts
#   the ensemble by the same constant.
# - Removes one degree of freedom: avoiding redundancy with a constant term and
#   modestly stabilizing weights under collinearity.
#
# Conclusions
# ###########

Review comment (suggestion):
Can the estimator variable more closely match the text?

```suggestion
super_learner = StackingRegressor(
```

Code after:
stacking_regressor.fit(X, y)
stacking_regressor.final_estimator_.coef_

# %%
# How to mimic SuperLearner with scikit-learn
# ###########################################
#
# The `SuperLearner` [Polley2010]_ is a stacking strategy implemented as `an R
# package <https://cran.r-project.org/web/packages/SuperLearner/index.html>`_, but
# not available off-the-shelf in Python. It is closely related to the
# :class:`~sklearn.ensemble.StackingRegressor`, as both train the meta-model on
# out-of-fold predictions from the base estimators.
#
# The key difference is that `SuperLearner` estimates a convex set of
# meta-weights (non-negative and summing to 1) and omits an intercept; by
# contrast, :class:`~sklearn.ensemble.StackingRegressor` uses an unconstrained
# meta-learner with an intercept by default (and can optionally include raw
# features via passthrough).
#
# Without an intercept, the meta-weights are directly interpretable as
# fractional contributions to the final prediction.

from sklearn.linear_model import LinearRegression

linear_reg = LinearRegression(fit_intercept=False, positive=True)
super_learner_like = StackingRegressor(
    estimators=estimators, final_estimator=linear_reg
)
super_learner_like.fit(X, y)
super_learner_like.final_estimator_.coef_

# %%
# The sum of meta-weights in the stacked regressor is close to 1.0, but not
# exactly one:

super_learner_like.final_estimator_.coef_.sum()

# %%
# Beyond interpretability, the normalization to 1.0 constraint in the `SuperLearner`
# presents the following advantages:
#
# - Consensus-preserving: if all base models output the same value at a point,
#   the ensemble returns that same value (no artificial amplification or
#   attenuation).
# - Translation-equivariant: adding a constant to every base prediction shifts
#   the ensemble by the same constant.
# - Removes one degree of freedom: avoiding redundancy with a constant term and
#   modestly stabilizing weights under collinearity.
#
# The cleanest way to enforce the coefficient normalization with scikit-learn is
# by defining a custom estimator, but doing so is beyond the scope of this