Repository: scikit-learn/scikit-learn
File: examples/ensemble/plot_stack_predictors.py (Python)

Code before:

    y_pred = cross_val_predict(est, X, y, n_jobs=-1)
    scores = {
        key: (
            f"{np.abs(np.mean(scores[f'test_{value}'])):.2f}"
            r" $\pm$ "
            f"{np.std(scores[f'test_{value}']):.2f}"
        )
        for key, value in scorers.items()
    }

    display = PredictionErrorDisplay.from_predictions(
        y_true=y,
        y_pred=y_pred,
        kind="actual_vs_predicted",
        ax=ax,
        scatter_kwargs={"alpha": 0.2, "color": "tab:blue"},
        line_kwargs={"color": "tab:red"},
    )
    ax.set_title(f"{name}\nEvaluation in {elapsed_time:.2f} seconds")

    for name, score in scores.items():
        ax.plot([], [], " ", label=f"{name}: {score}")
    ax.legend(loc="upper left")

plt.suptitle("Prediction errors of base versus stacked predictors", y=1)
plt.tight_layout()
plt.subplots_adjust(top=0.9)
plt.show()

# %%
# Even if the scores overlap considerably after cross-validation, the predictions
# from the stacked regressor are slightly better.
#
# Once fitted, we can inspect the coefficients (or meta-weights) of the trained
# `final_estimator_` (as long as it is a linear model). They reveal how much the
# individual estimators contribute to the the stacked regressor:

stacking_regressor.fit(X, y)
stacking_regressor.final_estimator_.coef_

# %%
# We see that in this case, the HGBT model is somewhat dominant, with the spline
# ridge also contributing meaningfully. The plain linear model does not add
# useful signal once those two are in place; with
# :class:`~sklearn.linear_model.RidgeCV` as the `final_estimator`, it is not
# dropped but gets a tiny negative weight that acts as a small correction to
# counter its residual bias.
#
# If we switch to :class:`~sklearn.linear_model.LassoCV` as the
# `final_estimator`, that small, unhelpful contribution is set exactly to zero,

Review comment (suggestion):

```suggestion
plt.suptitle("Prediction Errors of Base versus Stacked Predictors", y=1)
```

Code after:
    y_pred = cross_val_predict(est, X, y, n_jobs=-1)
    scores = {
        key: (
            f"{np.abs(np.mean(scores[f'test_{value}'])):.2f}"
            r" $\pm$ "
            f"{np.std(scores[f'test_{value}']):.2f}"
        )
        for key, value in scorers.items()
    }

    display = PredictionErrorDisplay.from_predictions(
        y_true=y,
        y_pred=y_pred,
        kind="actual_vs_predicted",
        ax=ax,
        scatter_kwargs={"alpha": 0.2, "color": "tab:blue"},
        line_kwargs={"color": "tab:red"},
    )
    ax.set_title(f"{name}\nEvaluation in {elapsed_time:.2f} seconds")

    for name, score in scores.items():
        ax.plot([], [], " ", label=f"{name}: {score}")
    ax.legend(loc="upper left")

plt.suptitle("Prediction Errors of Base versus Stacked Predictors", y=1)
plt.tight_layout()
plt.subplots_adjust(top=0.9)
plt.show()

# %%
# Even if the scores overlap considerably after cross-validation, the predictions
# from the stacked regressor are slightly better.
#
# Once fitted, we can inspect the coefficients (or meta-weights) of the trained
# `final_estimator_` (as long as it is a linear model). They reveal how much the
# individual estimators contribute to the the stacked regressor:

stacking_regressor.fit(X, y)
stacking_regressor.final_estimator_.coef_

# %%
# We see that in this case, the HGBT model dominates, with the spline
# ridge also contributing meaningfully. The plain linear model does not add
# useful signal once those two are included; with
# :class:`~sklearn.linear_model.RidgeCV` as the `final_estimator`, it is not
# dropped, but receives a small negative weight to correct its residual bias.
#
# If we use :class:`~sklearn.linear_model.LassoCV` as the
# `final_estimator`, that small, unhelpful contribution is set exactly to zero,
# yielding a simpler blend of the spline ridge and HGBT models.