Repository: scikit-learn/scikit-learn
File: examples/release_highlights/plot_release_highlights_1_8_0.py (Python)

Code before:
    n_classes=5,
    n_clusters_per_class=1,
    class_sep=2.0,
    random_state=42,
)
X_train, X_calib, y_train, y_calib = train_test_split(X, y, random_state=42)
clf = LinearSVC(random_state=42)
clf.fit(X_train, y_train)
ts = CalibratedClassifierCV(FrozenEstimator(clf), method="temperature", ensemble=False)
ts.fit(X_calib, y_calib)
beta = ts.calibrated_classifiers_[0].calibrators[0].beta_
print(f"Optimal temperature = {1 / beta:.3}")

# %%
# Linear models improvements
# --------------------------
# There are two main developments going on for linear models: efficiency and API
# changes.
#
# Efficiency of squared error based models with L1 penalty
# ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
# The first one is a massive improvement of efficiency, in particular a reduced fit
# time, for squared error based estimators with L1 penalty: `ElasticNet`, `Lasso`,
# `MultiTaskElasticNet`, `MultiTaskLasso` and their CV variants. The fit time
# improvement is mainly achieved by **gap safe screening rules**. They enable the
# coordinate descent solver to set feature coefficients early to 0 and not look at them
# again. The stronger the L1 penalty the earlier features can be excluded from further
# updates.

from time import time

from sklearn.datasets import make_regression
from sklearn.linear_model import ElasticNetCV

X, y = make_regression(n_features=5000)
model = ElasticNetCV()
tic = time()
model.fit(X, y)
toc = time()
print(f"Fitting ElasticNetCV took {toc - tic:.3} seconds.")

# %%
# API changes in logistic regression
# ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
# After the deprecation (version 1.5) and removal (now in 1.8) of the `multi_class`
# parameter in `LogisticRegression` and `LogisticRegressionCV`, the bookkeeping goes
# on. The goal for `LogisticRegressionCV` is to fix types and shapes of a few fitted
# attributes: `C_`, `l1_ratio_`, `coefs_paths_`, `scores_`, `n_iter_`. To ease the
# transition, you can choose between new and old types and shapes with the new and
# temporary parameter `use_legacy_attributes`.


Review comment (performance):
```suggestion
# coordinate descent solver to set feature coefficients to 0 early and not look at them
```
(nitpick: spell out small numbers, so "zero" instead of "0" but don't ask me to justify this style rule, I can't)

Code after:
#
# Free-threaded (also known as nogil) CPython is a version of CPython that aims
# to enable efficient multi-threaded use cases by removing the Global
# Interpreter Lock (GIL).
#
# For more details about free-threaded CPython see `py-free-threading doc
# <https://py-free-threading.github.io>`_, in particular `how to install a
# free-threaded CPython <https://py-free-threading.github.io/installing-cpython/>`_
# and `Ecosystem compatibility tracking <https://py-free-threading.github.io/tracking/>`_.
#
# In scikit-learn, one hope with free-threaded Python is to more efficiently
# leverage multi-core CPUs by using thread workers instead of subprocess
# workers for parallel computation when passing `n_jobs>1` in functions or
# estimators. Efficiency gains are expected by removing the need for
# inter-process communication. Be aware that switching the default joblib
# backend and testing that everything works well with free-threaded Python is an
# ongoing long-term effort.

# %%
# Temperature scaling in `CalibratedClassifierCV`
# -----------------------------------------------
# Probability calibration of classifiers with temperature scaling is available in
# :class:`calibration.CalibratedClassifierCV` by setting `method="temperature"`.
# This method is particularly well suited for multiclass problems because it provides
# (better) calibrated probabilities with a single free parameter. This is in
# contrast to all the other available calibrations methods
# which use a "One-vs-Rest" scheme that adds more parameters for each class.

from sklearn.calibration import CalibratedClassifierCV
from sklearn.datasets import make_classification
from sklearn.naive_bayes import GaussianNB

X, y = make_classification(n_classes=3, n_informative=8, random_state=42)
clf = GaussianNB().fit(X, y)
sig = CalibratedClassifierCV(clf, method="sigmoid", ensemble=False).fit(X, y)
ts = CalibratedClassifierCV(clf, method="temperature", ensemble=False).fit(X, y)

# %%
# The following example shows that temperature scaling can produce better calibrated
# probabilities than sigmoid calibration in multi-class classification problem
# with 3 classes.

import matplotlib.pyplot as plt

from sklearn.calibration import CalibrationDisplay

fig, axes = plt.subplots(
    figsize=(8, 4.5),
    ncols=3,
    sharey=True,
)