ICML 2025

A Unified Comparative Study with Generalized Conformity Scores for
Multi-Output Conformal Regression
Victor Dheur 1 Matteo Fontana 2 Yorick Estievenart 1 Naomi Desobry 1 Souhaib Ben Taieb 1 3
Abstract
Conformal prediction provides a powerful frame-
work for constructing distribution-free prediction
regions with finite-sample coverage guarantees.
While extensively studied in univariate settings,
its extension to multi-output problems presents
additional challenges, including complex output
dependencies and high computational costs, and
remains relatively underexplored. In this work,
we present a unified comparative study of nine
conformal methods with different multivariate
base models for constructing multivariate predic-
tion regions within the same framework. This
study highlights their key properties while also ex-
ploring the connections between them. Addition-
ally, we introduce two novel classes of conformity
scores for multi-output regression that generalize
their univariate counterparts. These scores ensure
asymptotic conditional coverage while maintain-
ing exact finite-sample marginal coverage. One
class is compatible with any generative model, of-
fering broad applicability, while the other is com-
putationally efficient, leveraging the properties of
invertible generative models. Finally, we conduct
a comprehensive empirical evaluation across 13
tabular datasets, comparing all the multi-output
conformal methods explored in this work. To en-
sure a fair and consistent comparison, all methods
are implemented within a unified code base1.
1Department of Computer Science, University of Mons, Mons,
Belgium 2Department of Computer Science, Royal Holloway, Uni-
versity of London, Egham, United Kingdom 3Department of Statis-
tics and Data Science, Mohamed bin Zayed University of Artificial
Intelligence, Abu Dhabi, United Arab Emirates. Correspondence
to: Victor Dheur <victor.dheur@umons.ac.be>.
Proceedings of the 42 nd International Conference on Machine
Learning,...