Repository: scikit-learn/scikit-learn
File: examples/ensemble/plot_stack_predictors.py (Python)

Code before:

import numpy as np
import pandas as pd

rng = np.random.RandomState(42)
X = rng.uniform(-3, 3, size=500)
trend = 2.4 * X
seasonal = 3.1 * np.sin(3.2 * X)
drop = 10.0 * (X > 2).astype(float)
sigma = 0.75 + 0.75 * X**2
y = trend + seasonal - drop + rng.normal(loc=0.0, scale=np.sqrt(sigma))

data = np.c_[X, y]
df = pd.DataFrame(data, columns=["X", "y"])
_ = df.plot.scatter(x="X", y="y")

# %%
# Stack of predictors on a single data set
# ########################################
#
# It is sometimes not evident which model is more suited for a given task, as
# different model families can achieve similar performance while exhibiting
# different strengths and weaknesses. Stacking combines their outputs to exploit
# these complementary behaviors and can correct systematic errors that no
# individual model fixes alone. With appropriate regularization in the
# `final_estimator`, the `StackingRegressor` often matches the strongest base
# model, and can outperform it when base learners' errors are only partially
# correlated, allowing the combination to reduce individual bias/variance.
#
# Here, we combine 3 learners (linear and non-linear) and use the default
# :class:`~sklearn.linear_model.RidgeCV` regressor to combine their outputs
# together.
#
# .. note::
#    Although some base learners include preprocessing (such as the
#    :class:`~sklearn.preprocessing.StandardScaler`), the `final_estimator` does
#    not need additional preprocessing when using the default
#    `passthrough=False`, as it receives only the base learners' predictions. If
#    `passthrough=True`, `final_estimator` should be a pipeline with proper
#    preprocessing.

from sklearn.ensemble import HistGradientBoostingRegressor, StackingRegressor
from sklearn.linear_model import RidgeCV
from sklearn.pipeline import make_pipeline
from sklearn.preprocessing import PolynomialFeatures, SplineTransformer, StandardScaler

linear_ridge = make_pipeline(StandardScaler(), RidgeCV())

spline_ridge = make_pipeline(
    SplineTransformer(n_knots=6, degree=3),
    PolynomialFeatures(interaction_only=True),

Review comment (suggestion):

```suggestion
# `final_estimator`, the :class:`~sklearn.ensemble.StackingRegressor` often matches the strongest base
```

modulo linting

Code after:

import numpy as np
import pandas as pd

rng = np.random.RandomState(42)
X = rng.uniform(-3, 3, size=500)
trend = 2.4 * X
seasonal = 3.1 * np.sin(3.2 * X)
drop = 10.0 * (X > 2).astype(float)
sigma = 0.75 + 0.75 * X**2
y = trend + seasonal - drop + rng.normal(loc=0.0, scale=np.sqrt(sigma))

df = pd.DataFrame({"X": X, "y": y})
_ = df.plot.scatter(x="X", y="y")

# %%
# Stack of predictors on a single data set
# ########################################
#
# It is sometimes not evident which model is more suited for a given task, as
# different model families can achieve similar performance while exhibiting
# different strengths and weaknesses. Stacking combines their outputs to exploit
# these complementary behaviors and can correct systematic errors that no single
# model can fix on its own. With appropriate regularization in the
# `final_estimator`, the :class:`~sklearn.ensemble.StackingRegressor` often
# matches the strongest base model, and can outperform it when base learners'
# errors are only partially correlated, allowing the combination to reduce
# individual bias/variance.
#
# Here, we combine 3 learners (linear and non-linear) and use the default
# :class:`~sklearn.linear_model.RidgeCV` regressor to combine their outputs
# together.
#
# .. note::
#    Although some base learners include preprocessing (such as the
#    :class:`~sklearn.preprocessing.StandardScaler`), the `final_estimator` does
#    not need additional preprocessing when using the default
#    `passthrough=False`, as it receives only the base learners' predictions. If
#    `passthrough=True`, `final_estimator` should be a pipeline with proper
#    preprocessing.

from sklearn.ensemble import HistGradientBoostingRegressor, StackingRegressor
from sklearn.linear_model import RidgeCV
from sklearn.pipeline import make_pipeline
from sklearn.preprocessing import PolynomialFeatures, SplineTransformer, StandardScaler

linear_ridge = make_pipeline(StandardScaler(), RidgeCV())

spline_ridge = make_pipeline(
    SplineTransformer(n_knots=6, degree=3),
    PolynomialFeatures(interaction_only=True),