ICML 2025

Algorithms with Calibrated Machine Learning Predictions
Judy Hanwen Shen 1 Ellen Vitercik 1 2 Anders Wikum 2
Abstract
The field of algorithms with predictions incorpo-
rates machine learning advice in the design of
online algorithms to improve real-world perfor-
mance. A central consideration is the extent to
which predictions can be trusted—while existing
approaches often require users to specify an aggre-
gate trust level, modern machine learning models
can provide estimates of prediction-level uncer-
tainty. In this paper, we propose calibration as
a principled and practical tool to bridge this gap,
demonstrating the benefits of calibrated advice
through two case studies: the ski rental and on-
line job scheduling problems. For ski rental, we
design an algorithm that achieves near-optimal
prediction-dependent performance and prove that,
in high-variance settings, calibrated advice offers
more effective guidance than alternative methods
for uncertainty quantification. For job scheduling,
we demonstrate that using a calibrated predictor
leads to significant performance improvements
over existing methods. Evaluations on real-world
data validate our theoretical findings, highlighting
the practical impact of calibration for algorithms
with predictions.
1. Introduction
In recent years, advances in machine learning (ML) mod-
els have inspired researchers to revisit the design of classic
online algorithms, incorporating insights from ML-based
advice to improve decision-making in real-world environ-
ments. This research area, termed algorithms with predic-
tions, seeks to design algorithms that are both robust to
worst-case inputs and achieve performance that improves
with prediction accuracy (a desideratum termed consistency)
(Lykouris & Vassilvitskii, 2018). Many learning-augmented
1Department of Computer Science, Stanford University, Stan-
ford, CA, USA 2Department of Management Science & Engineer-
ing, Stanford University, Stanford, CA, USA. Correspondence to:
Anders ...