Minimax Multi-Target Conformal Prediction with Applications to Imaging Inverse Problems
Authors: Jeffrey Wen
Rizwan Ahmad
Philip Schniter
Venue: Accepted by TMLR ()

Abstract: In ill-posed imaging inverse problems, uncertainty quantification remains a fundamental challenge, especially in safety-critical applications. Recently, conformal prediction has been used to quantify the uncertainty that the inverse problem contributes to downstream tasks like image classification, image quality assessment, fat mass quantification, etc.  While existing works handle only a scalar estimation target, practical applications often involve multiple targets. In response, we propose an asymptotically minimax approach to multi-target conformal prediction that provides tight prediction intervals while ensuring joint marginal coverage. We then outline how our minimax approach can be applied to multi-metric blind image quality assessment, multi-task uncertainty quantification, and multi-round measurement acquisition. Finally, we numerically demonstrate the benefits of our minimax method, relative to existing multi-target conformal prediction methods, using both synthetic and magnetic resonance imaging (MRI) data.

Published in Transactions on Machine Learning Research (11/2025)
Minimax Multi- Multi-Target Conformal Prediction with Applica- tions to Imaging Inverse Problems
Jeffrey Wen
wen. wh..kk..huckeeemmill..ooo
Deportment of Electricnl and Computer Engineering The Ohio Stote University
Rizwan Ahmad
rizwan, www../aaamma@@@uuss.....
Department of Biomedical Engineering The Ohio State University
Philip Schniter
scloiter. 1@lsterr.@@ou.ooo edu
Department of Electricol and Computer Engineering The Ohio State University
Reviewed on OpenReview: https:// opwnreveee..........mmmo net/ forum? 53FEYuDQKO
Abstract
In ill-posod imaging inverse problems, uncortainty quantification remains fundamental challenge, especially in safety-critical applications. Recently, conformal prediction has been used to quantify the uncertainty that the inverse problem contributes to downstream tasks like image classification, image quality assessment, fat mass quantification, etc. While existing works handle only scalar estimation target, practical applications often involve multiple largets. In response, we propose an asymplotically minimax approach to multi- target conformal prediction that provides tight prediction intervals while ensuring joint marginal coverage. We then outline how our minimax approach can be applied to multi- metric blind image quality assessment, multi-task uncertainty quantification, and multi-round measurement acquisition. Finally, we numerically demonstrate the benefits of our minimax method, relative to existing multi-target conformal prediction methods, using both synthetic and magnetic resonance imaging (MRI) data.
1 Introduction
Imaging inverse problems (Bertero et al., 2021 span wide array of tasks, such as denoising, impainting, accelerated magnetic resonance imaging (MRI), limited-angle computed tomography, phase retrieval, and image-ta-image translation. In such problems, the objective is to recover a true. image xp from noisy, incomplete, or distorted measurements 9o = A(zo). These problems tend to be ill-posed, in that many distinct hypotheses of Zo can explain the collected measurements Yu- When perfect recovery of To is difficult or impossible, uncertainty quantification (UQ) critical to safely using/ hitng/iiterpreting given reconstruction No. especially in high-stakes fields like science or modicine (Chu al., 2020: Banerji al., 2023).
The field of image recovery has evolved significantly over the decades, and most contemporary approaches are based deep learning (DL) (Arridge et al. 2019). Quantitatively, recent DL-based methods outperform classical methods on average and, qualitatively, they produce reconstructions that are sharp and detailed Ongie et al.) 2020). When the inverse problem is highly ill-posed, classical methods tend to produce recoveries with recognizable visual artifacts, from which it is relatively easy to gauge uncertainty. For example, radiologists receive explicit training this regard (Virmani et al. 2015).. In contrast, DL-basod methods can hallucinate, i.c., generate recoveries that are visually plausible but differ from the truth in clinically or scientifically important ways (Cohen et al. 2018 Belthangady & Royer 2019 Hoffman al. 2021 Muckley

---

Published in Transactions on Machine Learning Research (11/2025)
et al. 2021 Bhadra et al. 2021 Gottschling et al.
Tivnan et al. 2024 This underscores the need for rigorous UQ, e-g,, methods that provide statistical guarantees on estimates of xo or of some function (x(ro).
2023:
For example, a recent line of work (Wen et al. 2024 Cheung et al. 2024 quantifies the imaging-induced uncertainty on downstream tasks such as pathology classification or fat-mass quantification. Defining the target Zg as the output of the task applied to the (unknown) true image, they use conformal prediction (Vovk let al. 2005 Angelopoulos & Bates, 2023) to construct prediction intervals C. that are statistically guaranteed to contain the target. In a related line of work, Wen et al. (2025) provides statistical guarantees the quality of the reconstrueted image To relative to the true image, where "quality" is defined according to an arbitrary full-reference image-quality (FRIQ) metric like peak signal-to-noise ratio (PSNR) or structural similarity index measure (SSIM) (Wang et al. 2004). Defining the target as the FRIQ of To relative to the (unknown) true Zo. they use conformal prediction to construct a bound on FRIQ that is statistically guaranteed.
While the above methods rigorously quantify the downstream impact of roconstruction uncertainty, they handle only a scalar larget. In practice, one may want to consider multiple targets. For example, one may seek to identify multiple pathologies from single recovery or to judge the quality of that recovery according to multiple metrics. Although multi-target conformal prediction methods have been proposed, they suffer from either limited interpretability MMessoudi et al.
2022: Feldman et al., 2023) [Rosenberg et al.) 2023
Thurin et al. 2025 Klein et al. 2025 Braun et al.
2025), a lack of guaranteed joint coverage (eessoui
et al. 2021 Teneggi et al.] 2023
Park & Cho 2025 or some combination of overly conservative prediction
intervals and and//r /or high computational complexity (Messoudi et al. 2020 Diquigiovanni et al. 2022
Sampson
& Chan 2024
Sun & Yu 2024). as we explain in the sequel.
We thus propose new approach to multi-target conformal prediction. For problems with K 2 1 targets, our goal is to ensure a notion of fairness between largets. With prediction intervals C. and scalar targels Zo, for k ===la I....,,,, K, our method aims to ensure that no one "single-targe coverage" Pr E CC}} is favored over another, while also ensuring that all prediction intervals simultancously contain their corresponding targets with a user-specified probability of a-a. To do this, we minimize the maximum single-target coverage under the joint-coverage constraint Pr(ni Zo, E C&} S aa. Since the single-target coverage increases with the interval size [Cx], our approach additionally aims to prevent any prediction set C2 from being unnecessarily large. Our contributions are as follows:
Using a minimax formulation, we propose a novel multi-target conformal prediction approach with finite-sample marginal joint-coverage guarantees and low computational complexity.
2. We prove that our method is minimax in the limit of infinite tuming and calibration data.
3. For inverse problems, we propose a multi-round measurement acquisition scheme with marginal coverage guarantees on the final round.
We numerically compare our proposed method to several existing multi-target conformal prediction methods on a synthetic-data problem and four accelerated-MRR problems.
2 Background
2.1
Single-target conformal prediction
Conformal prediction (Vovk et al., 2005 Angelopoulos Bates 2023) is a general framework that enables one to construct uncertainty intervals with certain statistical guarantees for any black-box predictor. Importantly, it does not require any distributional assumptions the data other than exchangeability, which allows for adoption in broad range of applications. In this section, we briefly review the basics of conformal prediction, and in particular the computationally--ffiieie version known as split conformal prediction (Papadopoulos et al. 2002 Lei et al. 2018)
Suppose that we have a black-box model : 44 + RR that prodicts a target 2g E R from features uo E U. The prediction 2, = h(wo) may or may not be close to the true target 2n, but one can use conformal prediction compute a prediction interval Cx(2)) C IR that contains 20 with high probability. To compute this interval,
2

---

Published in Transactions on Machine Learning Research (11/2025)
conformal prediction uses dataset {(4,, 2)]= of centure-taree pairs distinct from those used to train h(.). using 2, = h(w4,, and deal is used to find a
This dataset is converted to calibration set deal _ X(dea)) satisfying the marginal coverage guarantee (Lei & Wasserman 2014
Pr Zo E ( aa
(1)
where a is a user-chosen error rale. Here and in the sequel, we use capital letters to denote random variables and lower-case letters to denote their realizations. In words, T guarantees that the unknown target Zo falls within the interval (ipm.) (Zo) with probability at least 1-a when averaged over the randomness in the test
data (Zo, Zo) and calibration data Deal-
The process of computing X(dea)) is known as calibration. To calibrate, one first defines nonconformity score s(2,, The choice of the nonconformity score function quite flexible; it requires only that the score is higher when there is worse agreement between & and ε, Common approaches include the absolute residual, docally-weighten residual (Lci et al., 2018), and conformalized quantile regression methods (Romano et al., 2019). The nonconformily score 8, (2 24) is then computed for each sample pair (2,, 2,) in the calibration set deal; and X(deat) is chosen as
(II-a)(x+ + 1)1 111,
((dea) A EmpQuant
881 ;18....,88)
n
which is a slightly more conservative quantile than the 1 1 quantile. With ((dea)) computed, the prediction interval for the ith sample is simply defined as
(idwa (2:) ((2,) {(deal))
Following this design, the marginal coverage guarantee 1 holds when (Zo, 71) (Zn, 7n) are statistically exchangcable (Vovk et al. 2005). weaker condition than i.i.d. Under the additional assumption that the nonconformily scores Sn, Sh.... Sn are almost surely distinct, the coverage can also be upper
bounded (Romano et al. 2019 by
Pr Pr E CXDD (20) S
+ n n+1
(4)
2.2 Application to imaging inverse problems
In imaging inverse problems, conformal prediction has emerged as a tool to quantify the uncertainty in image recovery. Several approaches (Angelopoulos et al.) 2022b Horwitz & Hoshen 2022 Teneggi et al.] 2023
Kutiel et al. 2023 Narnhofer et al. 2024) use conformal prediction to construct, for each individual pixel, an interval that guaranteed contain the true pixel value with high probability. For quantifying multi-pixel uncertainly, Belhasin et al. (0233 propose to compute conformal intervals the principal components of the
posterior covariance matrix, and Sannaarnaaaaaaaan et al. (2022) construct conformal intervals for semantic attributes in the latent space of a disentangled generative adversarial network.
Although these notions of uncertainty are interesting to consider, they don't directly quantify the impact of recovery errors on downstream imaging tasks such as image classification, image quality assessment, and quantitative imaging. Consequently, task-based image uncertainty methods like (Wen et al., 2024 Choung et al. 2024 Wen et al. 2025 have been proposed. We now briefly review these methods using unified notational framework.
Both Wen et al. (022)) and Cheung et al. 2024) quantify the uncertainty in estimating u(0o) E R given the measurements yo. In Wen et al. (2024) p(-)) is a soft-output classifier, and in Cheung et al. (2024) it is a fat-mass quantifier, but in either case the target is set at 2p m(z0)) Assuming access to an approximate posterior sampler x(s))) that generates samples (-."²) } ) per measurement. vector yi via x.(5) g(v,, )(()
using i.i.d code vectors 5(() ~ I), the prediction is computed as
Σ ( "...,2/-11
2 = la(a("), ( R
(5)
3

---

Published in Transactions on Machine Learning Research (11/2025)
Several nonconformity scores can be used, but here describe only the version with the conformalized quantile regression (CQR) method of Romano et al. 2019). With CQR, the and 1 - 2 empirical quantiles are computed as
g4 (2,) EmpQuant (g:20).... 0........ 2 and (2)) EmpQuant (1- 2 2 .
(6)
respectively, and the nonconformity score is defined as
s(2, 2;) = max (0g (2:) 24,2; -n
(7)
after which the prediction interval C Ad (2)) is constructed as in 13 Because this prediction interval changes with yis is said to be "adaptive"
Lei et al. 2018). In any case, satisfies the marginal coverage guarantee ZZ. Zm) are statistically exchangeable.
in when (Zo. Zo). (Z,, Zi).....
In related work, Wen et al. (2025) seek to estimate the FRIQ (c.g., PSNR, SSIM, etc.) m(Fu, mo) of an image recovery in (yo) relative to the true image xo when given access to measurements yo but not To itself. To do so, they set the target as 20 m(@0, ro) and use approximate posterior sampler that generates samples { per measurement vector y/ to compute the prediction
= [m( ) m(G,, (G,,,,1))1 1 E E..
Wen al. (2025) then used empirical quantiles to construct one-sided prediction interval (x)+) to either lower- upper-bouue bound the FRIQ, appropriate. Note that m(f.,-) can be viewed recovery-condtioone task. To maintain consistency with other task-based approaches, when discussing FRIQ estimation in the soquel, we construct two-sided intervals using
(3] with the nonconformity score from 7
2.3 Multi-target conformal prediction
As discussed in Section D one may be interested in conformal prediction of several targets, which we combine into a multi-dimensional target vector [2t.1...., xx,l = E RK We focus on the case where one is given a prediction E E R of unknown test 2g E &R.K along with a calibration set dar = ((2, =)) and the goal is to compute K prediction intervals {C (Cx(Da). (Z that satisfy the joint marginal coverage guarantee
Pr ( Zo.k E CXIDa), (20) 21
This guarantee ensures that all target components simultaneously lie within their respective prediction intervals with a probability at least 1 over the randomness in the calibration and test data.
Variations on (9] are possible, such as minimizing a risk that allows a fraction of the larget components lie outside their prediction intervals (Teneggi et al. 2023). Likewise, while (3) can be interpreted as constructing hyper-rectangle in RK that contains Zo with high probability, it is possible to construct non-rectangular regions, such as ellipsoidal regions (Messoudi et al. 2022) more complicated regions defined by the outputs of a conditional variational auto-encoder (Feldman et al. 2023), vector quantile regressor (Roeebbrgg et al. 2023), optimal-transport map (Thurin et al.) 2025 Klein et al. 2025), or a volume- moiume-inniizing mapping [Braun et al.) 2025). Although these non-rectan regions can give smaller uncertainty volumes, they are less interpretable for the applications we consider, since the uncertainty interval on one target component. will depend on the values of the other target components.
Inspired by 9 several approaches have been proposed to construct prediction intervals. Messoudi et al. 2020 statistically independent, that when the components
assume that the nonconformity score components are individually calibrated to yield an error-rate of a1, the joint error-rate a will equal 1 - (1-a))* This allows selting an = - (1 - 2// to meet a desired joint error-rate of a. However, the independence assumption may not hold in practice, where one could encounter K dependent score components that yield a
joint error-rate > 1 - (1 (1 ) in which case the joint-coverage guarantee (9) would be violatod. But even when the joint coverage holds, we show in Section 5 that the prediction intervals from Messoudi et al. (2020) are overly conservative.
4

---

Published in Transactions on Machine Learning Research (11/2025)
Another line of work uses copulas Neelsen 2006] to model the statistical dependency between the score
components { ] k=1 where 8i.k = (Zi,:: 2i,k). Given random vector 5 = [8],- ................................
.. Sx] with marginal
CDFs Fs, (sk) = Pr(Sk= I skt, the copula of S is defined as the function Cs : [0, 1 - [0, 1] for which
Cs(vi...., uk) Prs (Si) S UI,...,, ,.s....,ssss. (Sk) S uk}.
(10)
That is, the probability integral transform is used to convert each marginal Sy into a uniform random variable VA (Sx) and the copula Cs is the joint CDF of V [Vi...., ......., Vx]. Messoudi et al. (2021) approximate the copula as Os using the empirical marginal CDFs Fs, computed using the calibration data, and then search for a 7 E [0, I] for which Cs(o,.... 6) 2 1-a. The corresponding X(dea) = [ (o), F, [o) E R
then approxinately satisfies ((((ada))) 2 1 1-a, and du (deal) can be used to define a prediction interval for
2,6 via 33 However, they provide no coverage guarantees on these intervals. Sun & Yu (2024) and Park &
Cho 2025 instead search for a Ô E [0, l| for which
i = arg min E un such that Cs(o)= 2 1 a, k=1
(11)
eE[0,1]
after which they set A(dea) ( Sx (wx) E IK Park & Cho (0025) use semiparametric vine copulas and provide coverage guarantees only in the limit infinite calibration data. Sun & Yu (2024) use the empirical copula together with conformal marginal CDFs (Vovk et al. 2017) and obtain a finite-data coverage guarantee similar
(o)),...
But (II) provides no incentive for balancing coverage across targets (as we will see in Section 5), and there's no clear way to solve the non-convex optimization in [11) Even approximately
solving [1 is computationally expensive (e.g.. Sun & Yu 2024) gradient descent).
As an alternative, Diquigiovanni et al. (2022) and Sampson & Chan 2024 propose to combine the score components {8i,)) k= into a single score via
8: max (a.1,,.......................... 8i,K]
(12)
Using calibration {s with (2 and extending 3 to component-wise intervals
((da)) (2x) {2 sk (2, 2) ((dea)) k l.....,,K
(13)
the arguments from Vovk el al.
2005 imply that the joint-coverage guarantee 19 and upper bound (4) both
hold under the usual exchangeability assumption. However, SSamsso & Chan (2024) note that this approach can disproportionally favor the target components with larger nonconformity scores, causing the prediction intervals of the other components to be overly conservative. To mitigate this issue, they propose to scale the nonconformity scores to a common range. To do so, they first train a pair of quantile regressors T5 (+) and (i-).a()) that estimate the z and 1 1- quantile of Zi,k, respectively, for each (l,..., KK,, and then form the scaled nonconformity scores
n- (2) T5. (3))
81.k max (Qg.a(2) 2i,k, Ft, -- -
(14)
( T5
Scale relative to 1st target
where 2, = uy. This helps to balance the single-target coverages PrfZo.k E C(DD). (Zo)) across ls E {1...., K} while ensuring the joint-coverage guarantee 9 and upper bound In Section we show how to obtain a better balance.
In the broader scope of distribution-free UQ the (2022a) provide an alternative approach to handling multiple targets that uses multiple hypothesis testing. Although LTT is generally used for cases in which there are multiple notions of risk, our case involves only a single risk and thus LTT would provide a guarantee the form
(LTT) framework of Angelopoulos et al.
Pr Pr nk Zu E Cx(Da) M. (Zo) Deal 2 1 - 1a12211 21-6 5,
(15)
5

---

Published in Transactions on Machine Learning Research (11/2025)
where a, E (0, 1) are each user-selected error rates. In 15 the inner probability is over the randomness in the test data while the outer probability is over the randomness in the calibration data. Since the LTT guarantee takes a different form than ( the LTT procedure is not directly comparable to any of the previonsly mentioned multi-target methods. Also, the use of two uscr-selectable crror rates in (15) complicates the design. In this paper, we locus only on methods that provide joint-coverage guarantees the form (9)
3 Minimax multi-target conformal prediction
In this section, we propose a new approach to multi-target conformal prediction using a minimax formulation. We first formulate the minimax problem using random variables. Then we present the finite-sample version of our approach, which manilests as an instance of split conformal prediction. Finally we prove that the sinite-sample solution converges the solution of the original minimax problem as the number of samples grows to infinity.
3.1 Random variable perspective
To build intuition, we first consider the design of prediction sets when the targets and predictions are
modeled random variables Z [Z,...., Σk] R and Ż [Zi...., Σk], respectively. For the kth component, suppose that the nonconformity score function is s)((;-) and the prediction set is constructed C, (2x) _ (z en (Zs, 2) Gr), where & is a design variable. Then the "single-targel coverage" of the kth component will be
Pr E (2x) Pr Ze E= : S [R)) Pr on (2,, Zz) S S.
(16)
Using Sy & sx (2, Zz), we can write the single single-target coverage more succinctly as
Pr (5x S Gx) Fs. (Ge), where Fs, (Gx) is the CDF of Sg evaluated at Er. Similarly, the joint coverage of all K components will be
(17)
Pr ( ( Za E C, (2x) = Pr np Ss < S...
(18)
For given joint miscoverage rate of a, we'd like to find tuple (G,...[(x) that ensures
Pr Pr St S 2x) 21 a
(19)
But, in general, many Gkk will yield the same value of Pr(n S (c), and for some choices of (G...., [k), portion of the prediction intervals Cg (2x) can be overly conservative. Since larger Pr(S, S { generally corresponds to larger prediction interval C2 (7x) due their monotonically non- decreasing relationship, we propose to design prediction intervals that minimize the maximum single-target coverage while ensuring joint coverage, iee,
....... = arg min max Pr(S, S {+} s.t. Prin < 2 Gi...
(20)
Using 17 and the fact that the CDF is non-decreasing, we can restate 20 as
= arg min max Fs. (Gx) s.t. Pr(nk ( Grn..
(21)
and further restate it using An A Fs, (Gx) as
a Ak)).,A))= arg min max da (S( S Ap....x
(22)

---

Published in Transactions on Machine Learning Research (11/2025)
Although the solution to 22 may not be unique, it suffices to find single minimax (A,,..., Xx). Towards this aim, observe that Pr(nk= k= Fs, (St) < Xx is monotonically non-decreasing with respect to any dR- Thus, given any tuple (AI....,A) Ak) that satisfies the constraint in (22). the tuple (X'...,,X) X)) for X max A also
satisfies the constraint while simultancously yielding the same value the objective "max; A This implics
that, without loss of minimax optimality, we can reframe (22 as a search for a single parameter A:
2 = arg min X s.t. Pr(n Ps, (Sx) 11 2
(23)
Although 22 bears some similarities to the copula methodology in note that (22 uses max where
uses sum. Also, (2 can be reduced to simple one-dimensional search 23 whereas the non-convex involves an expensive multi-variable optimization.
3.2
Finite-sample case
We now adapt (23) to the practical where the Sk distributions are unknown and must be learned. For this purpose, we assume access to "tuning" data {(uG, (i) Jitttr that is distinct from the data {(us, )i) used for split conformal prediction and from the data used to train the predictor that generates 2,.
We propose to do the following for each target component h E {1....,Kl K)- First, we construct the set druna,k {(Ei,) == and compute the nonconformity score 8i,k for all samples in deune, Using these nonconformity scores, compute the empirical CDF Fs, (.), where
Fs. (5) K 81,k S C, i n+l, n Maune
(24)
Thune
Next, we compute the nonconformity scores 81 for all samples i in the calibration set deat, - {(Zi,t, 2i,k) 1= and apply the learned Fs.() (.) to obtain the transformed calibration scores
FiA A Fs, (8i,k) for == 1.....n
(25)
Because Bi E [0, 1], the scores (B, are implicitly normalized across & E {1, K}. Finally, we take the maximum across components,
8; 1...,,........
(26)
and from these (3:)__1 compute X(deat) in the same manner as
2
(daai) = EmpQuant (I0 (I0-a)(n
an + 1)]
81, 8n
(27)
The target-domain prediction intervals can then be constructed as
(Rass (2:) {2: Fs. (s (s.(, 2)) < ((dea)), k = 1=.,,,,,K K.
(28)
Since the tuning data used to construct Fs, (.) distinct from the calibration data, our method follows the framework of split conformal prediction. By taking the max of the transformed scores, we ensure the inclusion
of all the scores, and thus enjoy the finite-sample joint marginal coverage guarantee of g summarized in the following lemma:
Lemma 1. For any (0, 1), the prediction intervals (Cx ( from 28 obey
Pr Pr n Zo.A E CXDD (2o) 2 -
(29)
when (2o, Zo), (2,, Zi),... (Zn, Zn) are statistically eachangeable.
The guarantee in Lemma similar to that in Sampson Chan 2024 and stronger than that in Sun & Yu (0244) since the latter requires exchangeability of the test, calibration, and luning samples. (In fact the
7

---

Published in Transactions on Machine Learning Research (11/2025)
Algorithm Minimax-basel conformal prediction of test. target 20 RRK from feature vector uo E U
Require: Error rate a (0, 1).
Test. feature vector ug.
Prediction model h: 44 - RK
Data {(u,, m not used to train h(-). Nonconformity score functions Sp. R x R + IR for k = K.
1: Compute the prodictions /20 using 2, = h(u)).
2: for == K do
Compute the nonconformity scores {st,k) Pat using Si,k = su (Z 21,b).
3:
Compute the empirical CDF (-) using Fs. Hii 8i,k C, - Nunee }
nune
5:
Compute the transformed nonconformity scores {3r,0 using 3,,k = Fs, (s1,k). Compute the coaponent-maximine scores {s,) using 8; = max(3;,1..... ............ 8i,
Compute the threshold X(dea)) EmpQuant [1 [11aa)n+11)
31 ,1,...,3 3,
8: Compute the prediction intervals (ida)) ).k (2: = {z: Fs, (2,,,,,,,) x)) S ((deal) } for k = I....,K. K.
9: return Prediction interval (dda)). (20) on unknown 20,6 for each h = 1, I,..., K.
proof in Sun & Yu (2024) makes the stronger assumption that the test, calibration, and tuning samples are Li.d.) In other words, the finite-sample guarantee Lemma ] is conditional on the tuning samples whereas that in (un & Yu (2024) not. Furthermore, we demonstrate Section 5 that, as a result of our minimax formulation, our coverages tend be more balanced (across targets) than those of Sampson & Chan (2024)
and Sun & Yu (2024) which prevents prediction intervals from being unnecessarily large.
Algorithm 1 summarizes our proposed split-conformal prediction procedure.
3.3 Asymptotically minimax
Because the proposed finite-sample methodology uses the empirical CDF 124 in place of the true CDF Fs. and the empirical quantile 226- 27 in place of the true quantile in 23 one may wonder how the calibration parameter A(dea)) in (27) relates to the minimax Ā in 233. Below we show that X(deal) converges to 2 in the limit of infinite tuning and calibration data.
Theorem 2. For each target component k = I,...,K, K, suppose that the nonconformity scores { th are i.idd with CDF (.)(.) and, for T 2 max; Fs, (Sx), suppose that (r(-) is continuous and strictly increasing at the (1-a)-leved quantile of T. Then A(dea) from (27) converges to X from (23 almost surely and Thune
See Appendix for proof.
4 Applications of multi- nulti-target conformal prediction in imaging
we described several applications of conformal prediction to single-target UQ in imaging inverse problems. In this section, propose several applications of conformal prediction to multi-target UQ in imaging inverse problems.
In Section 2.2
We now establish a common notation that can be used across several applications. Consider a target vector 2; E R prediction matrix & E xe, and nonconformity score $t,k = (x 2r,6) for the kth target and ith sample. With CQR, the nonconformity score would be computed as
$1,b s (Z (t,b) max {Tg A (Zt,)) Zi,ko Zi,k ã M (2
(30)
with
gg. (Zi.)) = EmppQunt
a (1)
and क (((.))((.)) = EmpQuant (1 2i.
<(c)
(1)(1) (11)
s(c)
(31)
2 ,,k.... ........
27,6
8

---

Published in Transactions on Machine Learning Research (11/2025)
With this problem setup, the proposed minimax method from Section 3 any of the existing multi-targe target approaches discussed in Section 2.3 can be applied to generate prediction sets ((d)), (5)) for target indices k =...... K. When the joint-coverage guarantee in (9) holds, all K prediction sets will simultnneusly include the corresponding true-target values with a probability of at least a We emphasize that this guarantoc holds regardless of the quality of the approximate postcrior sampler used generate ( As this sampler gets worse; the prediction sets will gel looser but the guarantee will still hold. Below, we describe how to construct these quantities in different applications.
4.1 Multi-metric blind FRIQ assessment
We first consider blind FRIQ assessment, where the goal is to estimate the FRIQ of a reconstruction T. = f(()
relative to the true image To, given measurements yo = A(zo) but no direct access to To. Whereas Section 2.2 discussed the use of a single FRIQ metric, one may instcad be interested in assessing image quality according several FRIQ metrics, since different metrics may be complementary (Wang 2011).
Consider the case of K FRIQ metrics (m+(s.)) To extend the conformal prediction approach from Section 2.2 sct the target vector 2; (2.,,,.... x R with 21, (ā, and use (e posterior samples { Jj=1 to form the prediction matrix 2 = , E IR Rxx where E, =
( m. Fi, 52 )T
4.2 Multi-task uncertainty quantification
Now consider task-based UQ as described in Section 2.2 for the case of a single task. In practice, one may want to consider several tasks, such as classifying the presence/abss absence several different pathologies from a single image. To extend the based UQ method from Section 2.2 to K downstream tasks {uu ( we form the target vector as = [Zi.1,..., xi E RRK with 2i,x = (xx and use posterior samples { (=0) (5
to form the prediction matrix & = [a. x E IR Rxe, where 2 = ( (?
4.3 Multi-round measurement acquisition
In Wen et al. 2024], ; the authors propose a task-based multi-round measurement protocol where measurements are gradually collected until the conformal interval length falls below a user-specified threshold т. More precisely, at the end of each measurement round b E (1,..., B}. prediction 2! IR and conformal prediction (b])
interval (2)()) are constructed from the cumulative mcasurements y/dh Measurement collection stops if (2!") )l - (or if = B) but otherwise continues to the next round. The goal is to reduce measurement costs while guaranteeing that the collected measurements are sufficient for the task. This is especially useful in applications like accelerated MRI, where long scan times increase both patient discomfort and the likelihood of motion artifacts (Knoll et al. 2020
A limitation of the multi-round protocol from Wen et al. (0024) is that the marginal coverage guarantee holds for each round in isolation, but not for the multi-round protocol whole. In the sequel, refer to the multi-round protocol from Wen et al. (2024)
the "separate calibration" (SC) method, because it is implemented using bank of B independently calibrated conformal prodictors. That is, although the
SC method ensures that Pr E c(6l ( ) 2 1 1-a a for each round b assuming (7n, ) (Zn, 2 are

---

Published in Transactions on Machine Learning Research (11/2025)
exchangeable, we really desire that the multi-round coverage
B Σ Pr
P multi
{ 7o E (2)))) final round b}
(32)
b=1
Pr Zo E (2.) c (2) S m
B-1
Pr Zo cl c (2)5 || = t,
T...... (c("(
7}
b=2
+ Pr Zo E c1 (2)) | ||l3-||| (2) B
1 (
(33)
is at lcast 1-a. This is because, in practice, we don't apriori know which round will be the accepted round, and thus we must ensure coverage in all cases. To address this limitation, we note from (32 that
B Σ Pri Zo E c (2 final round =b} = =Pr( (r Zo E Cl (2 }
P multi 2
(34)
with equality due to the fact that Ebmi Pr(final round = b3 = 1. Thus, if
Pr fr m Zo E clbl (2))))))) 2 a-a
(35)
then it follows that Pmulti 2 1-a. Since (35) is special case of the joint marginal coverage guarantee (9)
can ensure (35) using multi-target conformal prediction techniques like the one proposed in Section
That said, the above B-round protocol handles only scalar E R, i.e., a single task. To extend it to L tasks {((()))/ we set the target vector as 2, = [ k E RK where K = BL and 2i, 1)+1 = (x;)
for all b. Then, for cach round b, we generate c postcrior samples (1) ) via =:!(() g(u!" :7 (010)) with i.idd w/1()) N(O, I) and form the predictions Σ = -,,,....,,,."" E Kxe with 21,(6-1)+
(n:000))
In ).-..).)( (_) (= When the joint-coverage guarantee in (9) holds, the prediction intervals for all tasks will simultancously contain their respective targets with probability at least 1 1-a a in the final measurement round.
5 Numerical experiments
We now numerically evaluate the proposed asymplotically minimax multi-Largel conformal prediction method from Section 3.2 along with the andependence-assunpian (IA)-based method from Messoudi al. (2020), the normaiiaoonna (QN)-based method from [Sampson & Chan (2024), and the copula-based " "CPTS"
method from Sun & Yu (2024), all described in Section 2.3 We compare all four methods using both syuthetic data and real-world accelerated-MMR data. For 7 (.)) and 7i -.)()). we use the empirical quantile estimator (6) for all MRI experiments in Section 5.2 but train a quantile regression (Koenker & Bassett 1978) model for the synthetic experiments in Section 5.1 For the IA method, we use the CQR nonconformity score from 7
with an adjusted error-rate of 01 1- -x) * to provide a joint-coverage rate of 1 1-a. For the QN method, we compute the nonconformity scores for each target using (14) before taking the max across targets (12) For the CPTS method, we use the CQR nonconformity score from ( but otherwise use the implementation settings from Sun & Yu (2024). Since minimax method can be applied with any nonconformity score " and one with the QN
sx(;)), ) we test one variation with the CQR score ( denoted as
score 14] denoted as "QN+Minimax."
For evaluation, we first randomly draw a fixed tuning dataset deune from the non non-training data. Since the joint guarantce g holds over the randomness in the calibration and test data, we perform T 10000 Monte Carlo trials, where in each trial t (1...., T} we randomly partition the remaining non-training data into a calibration set (t) with indices E [() and test set with indices E [es As coverage metric, we
10

---

Published in Transactions on Machine Learning Research (11/2025)
evaluate the empirical joint coverage
T
EJC A
Σ
II
1f E Midall). (E))),
(36)
rest
iETealld kE()....k)
where I (.) is the indicator function. Since a larger coverage generally indicates a more conservative prediction interval for a given nonconformity score, we also measure the empirical targett coverage for each target k:
ESC₁ A
Σ
1( E (
(37)
rest t=1
iETemm (1)
and, quantify the corresponding prediction-interva size, we measure the mean interval length:
MILK
Σ iETnel()
(xconnle) (2:)l-
(38)
T
Test i|]
5.1 Synthetic data
We start with synthetic talti- regression problem. The random data {2,} are constructed ii.d across
E 10. n + Ttune + nhest + Atrrinn with Z; [Zr,,, Zi,,, Zi,3]
Z 10U, 10 4,1
(39a)
Z 2U; +
(39b)
Z,3 = 0.1U? +
(39c)
U. ~ Uni and two cases of random 6, = [et,1 f,,,, ei,2, T. In the first case, 4, ~ N(10, 1), 612 Gamma(shape 1, scale = 1), and 6, ~ Exp(scale 1) are independent. In the second case, & is formed by stacking those three independent noise variables into a vector and multiplying by the Cholesky factor
0.8
0.7
Σ
0.8
1
0.4
(40)
0.7
0.4
1
to construct a correlated noise vector.
In either case, we generale Thtriin 10 000 training samples. For each target component k, we train two linear quantile regressors T3 (-) and @ 2 (-) (.) to estimate the 3 and 1 1- quantiles of Z,,s, respectively, from ε Ug. We generate Thtune 10 000 tuning samples estimate the empirical CDF for the minimax approach (see [24] and CPTS approach (see Sun & Yul (2024 E5.)))). Then, for each Monte Carlo trial t, generate = 10 000 calibration samples and fhtes = 5000 test samples. For fairness, the TA and QN methods use the tuning samples as additional calibration samples.
Empirical coverage: Table [ shows the EJC (36 for each method versus the desired joint coverage 1-a. There we see that, the independent-noisee case, all four methods perform nearly identically, with EJCs that almost exactly match the desired 1-a. In the corrclated-noise casc, however, the IA method yiclds an overly conservative EJC while the QN, CPTS, and minimax methods give EJC of almost exactly the desired 1-a. This behavior is not surprising, since the IA method assumes independent target components while the other methods do not.
Single-target coverage: Figure aa) and (b) plot the max ESC₁ and ming ESC, of each method versus the desired joint coverage 1-a for the independent- and correlated-noll cases, respectively. With independent noise, the QN and CPTS methods result in the largest ESC spread (i.c., max, ESC, min, ESC2), especially at 1-a-- 0.8. With correlated noise, the LA method yields much more conservative ESC than the other methods, while the QN and CPTS methods result in the largest ESC spread, especially for 1-a & 0.75.
11

---

Published in Transactions on Machine Learning Research (11/2025)
Table 1: Empirical joint coverage versus desired joint coverage 1 a for the synthetic experiment.
Noise Type
Method
0.70
0.75
0.80
0.85
0.90
0.95
Messoudi et al.
20/0
0.7000
0.7501
0.8001
0.8501
0.9000
0.9499
CPTS (Sun & Yu
2024
0.7002
0.7501
0.8001
0.8501
0.9001
0.9501
Independent
QN
(Sappso & Chan
2024
0.7002
0.7501
0.8001
0.8501
0.9000
0.9500
CQR+ (Ours)
0.7000
0.7500
0.8000
0.8500
0.9000
0.9501
QN+Minimxx (Ours)
0.7002
0.7502
0.8001
0.8501
0.9000
0.9501
IA MMessoudi et al.
2020
0.7583
0.7991
0.8393
0.8791
0.9187
0.9584
CPTS (Sun Yu
2024
0.7001
0.7502
0.8001
0.8501
0.9001
0.9502
Correlated
QN Sampson & Chan
2024
0.7001
0.7501
0.8001
0.8500
0.9000
0.9500
CQR (Ours)
0.7000
0.7501
0.8001
0.8500
0.9000
0.9500
QN QN+Minimax (Ours)
0.7001
0.7502
0.8001
0.8501
0.9001
0.9501
Sensitivity to quantile-estimatorr quality: We now investigate the effect of quantile-estimator quality, which can be difficult to guarantee in practice. To do this, we vary the number of training samples Merrann used to train the quantile regressor. Since, when Reain small, the performance can vary significantly over different draws of the training set, we run the experiment five times. Figure [c] plots the mean (across these 5 runs) of max; ESC₁ and ming ESC₂ versus Terrain E {50, 500, 1000, 2000} with 0.1 and correlated noise. The QN method shows a large ESC; spread for small values of itrain, while the other methods show robustness to Atrain-
Sensitivity to tuning samples: Since the CPTS and minimax methods rely on tuning samples, we now investigate FSC; as the number of tuning samples Aeune varied. (Recall that the TA and QN methods use these tuning samples as additional calibration samples.) Again we run the experiment five times, drawing new training and tuning sets each time. Figure [d) plots the mean (across runs) of max; ESC₁ and ming ESC, versus Ttuue E (50, 500, 1000, 2000, 5000, 10000} with a = 0.1 and correlated noise. As expected, the CPTS and minimax methods perform poorly when Puune is too small. For sufficiently large Neunee both the CPTS and minimax methods perform on par with QN. But to reach this level of performance, CPTS requires more samples than the minimax methods (in this case, 10 000 versus 5000).
In summary, the IA method is sensitive to noise correlation, while the other methods are not. Mcanwhile, the QN method is sensitive to poor quantile estimators (e.g., from too-few training samples) while CPTS and the proposed minimax methods are sensitive to too-few tuning samples. Since conformal prediction is often applied post-hoc, where one has significant control over the the training, CPTS and the proposed minimax methods can be advantagcous. We will see this behavior arisc in the subsequent MRI experiments, where we have little control over the quantile-est imator performance. Finally, Fig. suggests that the proposed minimax methods offer more balanced single- target coverages than CPTS.
process but no control over
5.2 Magnetic resonance imaging
We now compare all five methods on experiments with accelerated magnetic resonance imaging (MRI) Knoll et al. 2020: Hammernik et al. (023). MRT renowned for its ability to provide high-quality soft. tissue images without the use of harmful ionizing radiation. However, MRI scans are slow, which compromises patient comfort and throughput and can lead to motion artifacts. In accelerated MRI, the scan time is reduced by a factor of R by collecting only 1/R of the measurements required by the Nyquist sampling theorem. Doing so, however, leads to an ill-posed imaging inverse problem, where it is impossible to guarantee recovery of the true image Thus, for robust diagnoses, uncertainty quantification becomes important.
We follow the experimental setup of Wen et al. (2024), which uses the fon-pat-uuppeeseed subset of the multicoil fastMRI knoe datasct from Zbontar et al. (2018). This subsct contains 17 286 training images and 2188 validation images. To generate the accelerated measurements, the spatial Fourier domain, known the "k-space", is retrospectively subsampled with random Cartesian masks at acceleration rates
Data:
12

---

Published in Transactions on Machine Learning Research (11/2025)
IA (Messoudi et al., 2020)
QN (Sampson & Chan, 2024)
CPTS (Sun & Yu, 2024)
CQR+Minimax (Ours)
QN+Minimax (Ours)
1.00
1.00
Single Target Coverage
0.95
0.95
0.90
0.90
0.85
0.85
0.7
0.8
0.9
0.7
0.8
0.9
1-a a
a
(a) Independent
(b) Correlated
1.00
1.00
Single Target Coverage
0.98
0.98
0.96
0.96
0.94
0.94
0.92
2500
5000
7500
10000
2500
5000
7500
10000
Number of Training Samples
Number of Tuning Samples
(c) Training Sensitivity
(d) Tuning Sensitivity
Figure 1: For the synthetic experiment, (a) shows min; ESC, and max; ESC; versus desired joint coverage 1 for the independent-non correlated-noise case, (c) shows ming FSC₁ and max; ESC₁ versus Titrain at. 1 --a 0.9 and (d) shows the same versus Nuune The traces in (c) and (d) represent the average across 5 draws of training and tuning data. In some cases the green curves are hidden behind the red curves.
case and (b) shows the same for the correlated-noise case. Then for the
13

---

Published in Transactions on Machine Learning Research (11/2025)
Table 2: Empirical joint coverage versus 1 1-a for the multi-metric and multi-task MRI experiments at R 8.
Task
Method
0.70
0.75
0.80
0.85
0.90
0.95
Messoouii et
2020
0.7797
0.8140
0.8485
0.8856
0.9264
0.9625
CPTS (Sun & Yu
2024
0.7025
0.7518
0.8025
0.8525
0.9026
0.9520
multi-metric
QN
(Sampon & Chan
2024
0.7006
0.7502
0.8006
0.8503
0.9005
0.9503
CQR (Ours)
0.7008
0.7511
0.8005
0.8506
0.9002
0.9506
QN+Miniaxx (Ours)
0.7010
0.7502
0.8007
0.8508
0.9002
0.9508
IA [Messoudi et al.
2020
0.7935
0.8344
0.8677
0.8997
0.9340
0.9697
CPTS (Sun
2024
0.7019
0.7529
0.8034
0.8518
0.9010
0.9526
multi-task
QN Sampson & Chan
2024
0.7005
0.7502
0.8005
0.8502
0.9005
0.9505
CQR +Mimmmx (Ours)
0.7013
0.7506
0.8009
0.8511
0.9004
0.9507
QN +NMMiniaxx (Ours)
0.7013
0.7505
0.8006
0.8509
0.9004
0.9505
R E (16,8, 10,,8,44 4, The masks use Golden Ratio Offset (GRO) sampling (Joshi et al. sampled autocalibration signal (ACS) region in the center, and they are nested such that the measurements collected at each R include all measurements collected at higher R. Soe Wen et al. (2024) for details.
2022 and include a fully
Models:
For the image-recovery model f(),, we use the popular F2FF-Varee from Sriram al. (2020), and for the posterior-sampling method 9(:-),, we use the conditional normalizing flow (CNF) from Wen et al.
(2023) with e 32 posterior samples. Both networks are trained (using the fastMRI training images) to handle acceleration rates R E {16, 8, 4, 2} following the procedure in Wen et al. (2024). The conformal predictors are all given access to the same tuning and calibration samples {(2,, and thus the image-recovery and posterior-sampling models are used identically across methods.
Validation: We first construct a tuning set diune using 656 of the 2188 fastMRI validation samples (i.e., 30%), selected randomly. Since the joint-coverage guarantee g holds over the randomness in the calibration and test data, we evaluate performance using T = 10 000 Monte Carlo trials. In each Monte Carlo trial I E (1...., T), we randomly partition the remaining validation data into calibration set daa(t] of size n = 1073 (or 50%) and a test set of size flest = 459 (or 20%) using indices E [esttt)- Because the 1A and QN methods do not use a tuning set, we add the tuning samples to their calibration sets (now of size n Tuune 1729) for fair comparison to the CPTS and minimax methods.
5.2.1 Multi-metric blind FRIQ assessment
We begin with the multi-metric blind FRIQ assessment problem from Section 4.1 consider PSNR, SSTM, learned perceptual image patch similarity (L.PIPS) ZZhng et al. 2018), and deep image structure and texture similarity (DISTS) (Ding et al.) (020).. The average FRIQ performance of the E2E-VarNet and CNF on the fastMRI validation set match those reported in Wen et al. (2025).
For the FRIQ metrics, we
Empirical coverage: Table 2 shows EJC versus desired joint coverage 1 1 at acceleration R 8. While all methods satisfy the joint-coverage guarantee g the EJC of the IA method is overly conservative.
Single-target coverage: Figure 2 plots max; ESC, and ming ESC₁ versus 1-a at R = 8. The QN method suffers from very highly spread ESCR, reminiscent of the synthetic experiment with poor quantile estimates, while the IA method suffers from overly conservative ESC,, reminiscent of the synthetic experiment with correlated noise. The CPTS method gives max; ESC₁ values similar IA (i.c., overly conservative) but slightly better for small values of 1-a, while the CPR-minimax method gives noticeably better max; ESC, than CPTS, which is consistent with its minimax formulation. (Recall that CPR-minimax and CPTS use the same nonconformity score.) Figure 3 shows ESC, individually for each target k; revealing that QN provides a massively conservative ESC, x for PSNR and an overly small ESC, for LPIPS. Based on Section 5.1 we conjecture that QN's coverage imbalance stems from unreliable quantile estimators 7 (.))(.) and (-) When the QN score is used within our minimax framework, however, the single-target coverages become well balanced.
14

---

Published in Transactions on Machine Learning Research (11/2025)
IA (Messoudi et al., 2020)
CPTS [Sun Yu, 2024)
QN (Sampson Chan, 2024)
Minimi (Durs)
IA [Meseoudi wt al., 2000)
ON (Sampson Chan, 2024)
Targets
QN-Minimax (Qurs)
CPTS (Bun & Yu, 2024)
+ CQR-Minima. (Ours)
+ QN-Minima (Ours)
PSNR 2: SSIM
LPIPS
Single Target Coveraige
1.00
Single Target Coverage
1:00
a 0.9
o 0.8
a 0.7
0.95
0:92
0.90
0.90
0.90
0.90
0.45
0.85
0.8
0.45
0.00
0.80
0.73
Target
Target
Target
0.7
0.9
1
Figure 3: ESC₁ for each FRIQ metric and desired coverage level 1-a in MRI at acceleration R = 8.
Figure 2: Min and max ESC₁ versus I-a for multi-metric MRI at acceleration R =
(Messouddi el., 2020)
(Sun 2024)
(Sempson Chan,
CQR Minimax (Oura)
QN+Minimax (Ours)
PSNR
SSIM
DISTS
LPIPS
Moan interval Langel Langth
14
0 275
0.275
12
0.14
250
10:
0.250
0.10
0.225
0 200
0.08
0.200
0.5
0.06
0.175
0.7
0.9
0.7
07
0.8
0.9
1-a a
1-a
1-a
1-
Figure 4: MIL versus 1 for the multi-metric MRI experiments at acceleration R = 8.
Mcan interval length: Figure 4 shows mean interval length MIL,; versus desired joint coverage 1-a for each metric k at R=8 8. For the PSNR metric, we see QN producing extremely loose prediction intervals at all 1-a, which is consistent with the overly generous single-target coverage shown in Fig. 3 For the LPIPS metric, although QN gives tighter prediction intervals than ON+Minimx, they come at the cost of very weak single-target coverage guarantees, as shown in Fig. 3 Although the IA method performs very consistently across the four target metrics, both CPTS and the proposed CQR+Mininax give lighter predictions intervals in every instance. Although CQR Minimmxa performs similarly to CPTS in most cases, gives noticeably tighter prodiction intervals for SSIM and LPIPS at high 1-a.
Overall, see the minimax methods behaving as expected in this multi-metric MRT experiment, i.e., providing well-balanced single-target coverages and thereby avoiding overly large prediction intervals.
5.2.2 Multi-task uncertainty quantification
We now consider a multi-task UQ problem, where the goal is to ascertain the presence/absenco of cach of 5 different pathologies accelerated MRI with R = We assume that multi-label soft-output classifier (a-) has been trained on clean images 2; output a vector of probabilities 2; E [o, 1] At inference time, since we have access only the accelerated measurements yo and not the true image Io, the goal is to construct, for each pathology k, prediction interval Mida) & (2o) that contains the true soft-output
F0,k [u(=o)le with some probabilistic guarantee. We apply the IA, QN, CPTS, Minimxx and QN- Minimxx methods as described in Section 4.2
For the classifier, we use a ResNet-50 (He et al. 2016) with K = 5 outputs in the final lincar layer. To train it, we first initialize using ImageNet weights, then pretrain using SimCLR loss (Chen et al. 2020 on the (unlabeled) fastMRI knee data, and lastly fine-tune using binary cross-entropy loss the (labeled) fastMRT+
15

---

Published in Transactions on Machine Learning Research (11/2025)
IA (Messoudi at. 2020)
CPTS (Sun Yu, 2024)
QN [Sampson Chan, 2024)
IA [Mesoodd et al, 2020)
QN (Sampson & Chan, 2024]
CQR-Minimax (Ours)
CPTS (Bun & 2024)
CQ (Cun)
+ QN-Miimme (Ouns)
Minimax (Ours)
0.9
0.8
1-u 0.7
Single Torgat Coverage
100
Singla Target Coveraga
0.92
0.55
0.5
0.95
0.40
0.90
0:83
0.85
Torget
Target
Target
0.85
Manisous
Carllage- Thicknass Losa Done Subchondral Edema
0.7
0.8
0.9
Bore-fraccure Contusion
Figure 5: Min and max ESC₁ versus 1-a for tulti-task MRI at acceleration R = 8.
Figure 6: ESC₁ versus target i for several values of 1-a in multi-task MRT at acceleration =
/Measoud nl. 2020)
CPTS (Sun 2024)
ON (Sempxon Chen, 2014)
CDR+ Minimax (Oung)
+ QN-Kiiimme (Ouni)
Cartilage-
Bone-Frecture Contusion
Edema
Mean Interval Length
0.14
0.35
0.20
0.12
0.08
0.30
0.30
0.15
0.68
0.28
0.00
0.06
0.04
0.7
0.8
0.3
0.7
2.8
0.9
0.7
0.8
0.9
0.7
0.5
0.9
0.7
0.2
0.9
Figure 7: MILa versus 1-a for each task k in multi-task MRI at acceleration R = -.
knee data Zhao et al. 2022 The chosen pathologies, listed in Fig. were the five with the most fastMRI+ samples. Additional details can be found in Appendix B
Empirical coverage: Table reports EJC versus desired joint coverage 1-a. There satisfy the joint-coverage guarantee (9) but that the EJC of IA is overly conservative. In fact, the IA method is even more conservative in this multi-task experiment than it was in the multi-metric experiment, which (recalling Section 5.1 may be due to increased correlation among the non-conformity scores for different tasks.
that all methods
Single-target coverage: Figure 5 plots the max; ESC, and ming ESC₁ versus 1-a for each method at R 8. Again, the 1A method gives overly conservative ESC₁ for all 1-a. The CPTS method suffers from imbalanced ESC, across targets, and consequently its max; ESC, is similar to IA's in most cases. By contrast, the CQR+ Cinimin method yields well-balanced ESC, over the entire range. As for the methods that use the QN nonconformity score, we see that QN and *N+Mmniax give similar FSC₁ except around 1-a 0.8, where +Nnmmxax is better balanced. Figure 6 shows ESC₁ individually for each target k and shows that each method distributes coverage across targets in a unique manner, especially at 1 1 0.7.
Mean interval length: Figure 7 plots MII.& versus desired joint coverage 1 for each target k. Among the CQR-based methods (i.e., 1A, CPTS, CQR+Minimax)), we see that IA tends to give the loosest prediction intervals, while CPTS and CQR+ trade for the tightest interval 1-a, however, CPTS gives unnecessarily loose intervals for of the classes, while CQR+ avoids this unwanted behavior. Relative to the CQR-based methods, the QN-based ones are much less consistent, in that they give very tight intervals for some classes (e.g., Meniscus Tear) but very loose ones for oers (e.g,, Bone-Subeer Subchondral Edema). Overall, QN and QN+Minimax achieve similar performance, with one or the other winning on caso-by-case basis. The loose interval produced by +inimxxxx at 1-a 0.95 most likely due the use of relatively few tuning samples (recall Fig. d)), recalling that the proposed methods minimax only asymptotically.
case- case-by-case by-case basis. For large
16

---

Published in Transactions on Machine Learning Research (11/2025)
5.2.3 Multi-round measurement acquisition with FRIQ guarantees
We now consider applying the multi-round measurement protocol from Section 4.3 to accelerate MRI while providing a probabilistic FRIQ guarantee. We adopt. the experimental setup of Wen et al. (2025), where measurements are collected over B rounds at acceleration rates R {16, 8, 4, 2, 1] but stop as soon conformal upper bound DIST falls below threshold of 0.16. To adapt the proposed multi- larget method from Section 4.3 to this upper-bounding setup, run the IA, CPTS, and +
methods with the one-sided CQR nonconformitt score 8% (2i,, 21,k) Zi, Φ 2 (21,)) and we run QN and
QN+ Minimaxx with one-sided QN nonconformity score
91 (1-).1(2))
F3 (2,)
8k (k,,,, z1,) = (zi,k 7- g.)))(()))
(41)
n1 g2)) - Og K (2,)
as motivated by Figure a) plots the empirical accepted coverage
T FAC A T Tese
Σ (ETuaal)]
1z E clu (2!")))
(42)
where by denotes the accepted round for the ith sample, versus the desired accepted coverage of 1-a for the IA, QN, CPTS, CQR++ and +N+Minimax versions of the multi-target method proposed in
Section 4.3 as well as the soparate-calibration (SC) method from Wen et al. (2024) discussed in Section 4.3
The figure shows that the measurements accepted by the SC method do not provide the desired coverage,
while those accepted by the target methods do, validating the goal of Section 4.3 Figure 8 b) plots the average accepted acceleration-rate
Ravg
1 T Σ
T
iEZura[ Ro. 111)
Σ
(43)
[t]
versus a-a, where Rs, is the acceleration rate at the accepted round for test sample i. Although the SC mothod achieves higher Ravg than the multi-target methods, it comes at the cost of not providing a coverage guarantee on accepted samples. Among the multi-target methods, the CPTS and CQR+Mimmmax methods yield the highest Ravg in all cases, demonstrating the advantage of tight conformal bounds. However, the much lower computational complexity of Cin-Minimax makes it advantageous in practice.
5.2.4 Multi-round measurement acquisition with downstream classification guarantees
Finally, we consider an application of the multi-round measurement protocol from Section 4.3 that aims to accelerate MRI while providing a probabilistic guarantee on downstream classification of multiple pathologies. For this we combine the multi-round setup from Section 5 5.2.3 with the multi-label sctup from Section 5.2.2 In particular, we lake measurements over B = 5 rounds at acceleration rates R E {16, 8, 4, 2, 1) but stop soon the prediction interval lengths for all T, pathology labels fall below 3 = 0.1. Rather than exclusively considering L = 5 pathology labels, we experiment with L E 3, 4,5) to investigate the effect of L. Here, L == I corresponds to the I most prevalent classes in the fastMRI+ training data.
For a desired accepted coverage of 1-a = 0.9, Fig. M(a) plots FAC versus the number of labels L. There we see that all methods meet the desired coverage for all L, which shows that the multi-round methodology proposed in Section 4.3 is flexible with regards to the choice of conformal predictor. Figure FigureXbb plots Ravg versus the number of labels T. for different conformal predictors. This figure shows that the proposed CQR- method yields the highest Raug across all tested values of L. It also shows that Ravg decreases with L, which is expected because the stopping criterion becomes more strict with larger L.
A recent clinical MRI study (Kastryulin et al. 2023 evaluated 35 FRIQ metrics and found that DISTS correlated best with radiologists' ratings of perceived noise level, contrast level, and artifacts when comparing reconstructions ground-truth images. 0.16 bocause this is the DISTS threshold at which single-round mcasurement scheme at
WWen ct al. (2025 chose accolcration It = achieves 1 a 0.95 coverage.
We chose 0.1 purely for the sake of demonstration. practically meaningful could he attained with expert guidance through clinical trials.
17

---

Published in Transactions on Machine Learning Research (11/2025)
(Messoudi at.,, 2020) -a- QN (Samppon & Chinn, 2024)
MiniMnn (Ouna)
-#- IA (Mesacudi et el., 2020) -a- QN (Sampaon Chan, 2024)
Destred
QNeMinniax (Ours)
SC (Wen
2025)
CPTS [Sun & Yu, 2024) -- Dasined
CQR+Miniax (Ours)
-+- ON-Miiiiieeeeee (Duns)
-a CPTS (Sun & Yu, 2024)
Accepted Coverage
Accepted Acc Acceleration
4.5
Accepted Caveroag
Accested Acceloration
0.9
4.0
6
0.96
0.8
0.94
3.0
0.92
0.90
1
1-
Number of Lobels L
Number of Lebels L (b)
(a)
(b)
(a)
Figure 8: For multi-round measurements that stop soon as the DISTS upper bound falls below 7 0.16, (a) plots EAC and (b) plots Ravg versus the desired accepted coverage 1-a.
Figure 9: For multi-round measurements that stop soon as the prediction intervals for all pathology labels fall below t = 0.1, (a) plots EAC and (b) plots Raug versus the number of labels L at 1 1-a = 0.9.
6 Conclusion
Motivated by the need for multi-target uncertainty quantification in imaging inverse problems, we propose a munimax-based approach to multi-target conformal prediction. The proposed method aims to minimize the maximun single-target coverage, max; Pr(Zu,: E (DDaii), (Zo)), across targets k subject a marginal
joint-coverage guarantee of the form Pr(nk=1Zo. Z0 E(DD..)).A (2)) 2 1 a- where a is user-specified. Because our approach is an instance of split conformal prediction, it guarantees marginal joint-coverage with finite-sized tuning and calibration datasets under the usual test /ett/ibbbbaiionn exchangeability condition. Furthermore, it converges to the minimax solution the size of the tuning and calibration sets grow to infinity. In addition to our minimax multi-target conformal predictor, we propose a multi-round measurement acquisition scheme that guarantees marginal coverage of the final-round prediction interval. We numerically compared the proposed minimax turgi- predictor to several existing methods on a synthetic-data problem as well as four accolcrated-MRI problems and found that the proposed minimax method gives better balancod single-target coverages while guaranteeing joint marginal coverage. In addition, we numerically investigated the proposed multi-round measurement scheme and confirmed that it provides marginal accepted coverage when used with variety of conformal predictors.
Limitations
There are several limitations to this work. First, like with many conformal prediction methods, the joint- coverage guarantee (9 holds only for prediction/target pairs Zz) that are statistically exchangeable over the test and calibration data. Furthermore, to prove that our approach is asymptotically minimax, we assumed that the nonconformity scores {St. 1 are i.i.d. Further work is needed to generalize these rest and the works Tibshirani et al. (011)) Barber et al. (022).. Cauchois et al. (2024) suggest. modifications that address non wor-escchage In addition, the proposed applications to MRI are preliminary, in that rigorous clinical trials are needed before they are adopted in practice.
Broader impact statement
We expect that our methodology will positively impact the field of imaging inverse problems by providing prediction intervals multiple estimation targets that involve the (unknown) true image. These intervals inform the practitioner of how much uncertainty the grasurement-nnd-roonsttueee downstream lasks, and whether the collected measurements are sulficient for a given reconstruction moth Furthermore, the proposed multi-round acquisition protocol allows one to collect fewer measurements while still providing guarantees on estimation performance. However, clinicians must be careful when interpreting the results, understanding, for example, that our coverage guarantees are marginal and not conditional. As such, they hold only when averaged over many different. test samples and calibration sets, rather than for
process introduces to
18

---

Published in Transactions on Machine Learning Research (11/2025)
specific test sample and/or calibration set. Furthermore, they hold only when the test sample is statistically exchangeable with the calibration samples.
Acknowledgments
This work supported in part by the National Institutes of Health under Grant R01-EB029957.
19

---

Published in Transactions on Machine Learning Research (11/2025)
References
Anastasios N. Angelopoulos and Stephen Bates. Conformal prediction: A gentle introduction. Foundations and Trends in Machine Learning, 16(4):494 591, 2023. ISSN 1935-8237. doi: /015611/200000101.
Anastasios N. Angeloponlos, Stephen Bates, Emmanuel J. Candès, Michael I. Jordan, and Lihua Lei. Learn then test: Calibrating predictive algorithms to achieve risk control. arXiv.2110.01005 2022a. doi: 00.445500/axXiv.1110.000
Anastasios N. Angelopoulos, Amit P. Kohli, Stephen Bates, Michael 1. Jordan, Jitendra Malik, Thayer Alshaabi, Srigokul Upadhyayula, and Yaniv Romano. Image-to-image regression with distribution-fire uncertainty quantification and applications in imaging, In Proc. Intl. Conf. on Machine Learning, 2022b.
doi: 10.48550/ars www.vv.222.........m
Simon Arridge, Peter Maass, Ozan Öktem, and Carola-Bibiane Schönlieb. Solving inverse problems using data-driven models. Acta Numerica, 28:1 174, June 2019.
Christopher R. S. Banerji, Tapabrata Chakraborti, Chris Harbron, and Ben D. MacArthur. Clinical Al tools must convey predictive uncertainty for each individual patient. Nature Medicine, 29(12):2996-290) 29988, 2023. doi: 0 (4008///44151--02-22260---7
Rina Foygel Barber, Emmanuel Candes, Aaditya Ramdas, and Ryan J Tibshirani. Conformal prediction beyond exchangeability. Annals of Statistics, (1)2):816 845, 2023.
Omer Belhasin, Yaniv Romano, Daniel Freedman, Ehud Rivlin, and Michael Elad. Principal uncertainty quantification with spatial correlation for image restoration problems. IEEE Trans. on Pattern Analysis and Machine Intelligence, 46:3321 16:3321-3333, -3333, 2023.
Chinmay Belthangady and Toic Royer. Applications, promises, and pitfalls deep learning for fluorescence image reconstruction. Nature Methods, 16(12):1215 1225, 2019.
Mario Bertero, Patrizia Boccacci, and Christine De Mol. Introduction to Inverse Problems in Imaging. CRC press, 2021.
Sayantan Bhadra, Varun A Kelkar, Frank Brooks, and Mark Anastasio. On hallucinations tomographic image reconstruction. TEEE Trans. on Medical Imaging, 40(11):3249-332) -2000, 2021.
Sacha Braun, Eugène Berta, Michael I. Jordan, and Francis Bach. Minimum volume conformal sets for multivariate regression. 0rXiw:2503.19068 2025.
Maxime Cauchois, Suyash Gupta, Almur Ali, and John C Duchi. Robust validation: Confident predictions even when distributions shift. Journal of the American Statistical Association, pp. 1-66, 2024.
Ting Chen, Simon Kornblith, Mohammad Norouzi, and Geoffrey Hinton. simple framework for contrastive learning of visual representations. In Proc. Inil. Conf. Machine Learning, pp. 1597-1607, 2020.
Matt Y Cheung, Tucker Netherton, Laurence E Court, Ashok Veeraraghavan, and Guha Balakrishnan. Metric-guided image reconstruction bounds via conformal prediction. arXiv:2404.1527, 2024.
Linda Chu, Anima Anandkumar, Hoo Chang Shin, and Elliot K Fishman. The potential dangers of
artificial intelligence for radiology and radiologists. Journal of the American College of Radiology, 17(10): 1309-1311 2020.
Joseph Paul Cohen, Margaux Luck, and Sina Honari. Distribution matching losses can hallucinate features in medical image translation. In Proc. Intl. Conf. on Medical Image Computation and Computer-Assistte Intervention, pp. 529-536, 2018.
Jia Deng, Wei Dong, Richard Socher, Li-Jia Li, Kai Li, and Li Fei-Fei, Imagenet: A large-scale hierarchical image database. In Proc. IEFE Conf. Compuler Vision and Pattern Recognition, pp. 248 255, 2009.
20

---

Published in Transactions on Machine Learning Research (11/2025)
Keyan Ding, Kede Ma, Shiqi Wang, and Eero P Simoncelli. Image quality assessment: Unifying structure and texture similarity. TEEE Trans. on Pollern Analysis and Machine Intelligence, 44(5):2567-255) -581,, 2020.
Jacopo Diquigiovanni, Matteo Fontana, and Simone Vantini. Conformal prediction bands for multivariate functional data. Journal of Multivariate Analysis, 189:104879, 2022.
Shai Feldman, Stephen Bates, and Yaniv Romano. Calibrated multiple-output quantile regression with representation learning. Journal of Machine Learning Research, 24(24):1-48, 2023.
Bert E Fristedt and Lawrence F Gray. A Modern Approach to Probability Theory. Springer, 2013.
Nina M Gottschling, Vegard Antun, Anders C Hansen, and Ben Adcock. The troublesome kernel On hallucinations, no free lunches and the accuracy-stability trade-off in inverse problems. arXiu:2001.01255
2023.
Kerstin Hammernik, Thomas Küstner, Burhaneddin Yaman Zhengnan Huang, Daniel Rueckert, Florian Knoll, and Mehmet Akçakaya, Physics-driven deep learning for computational magnetic resonance imaging: Combining physics and machine learning for improved medical imaging. IEEE Signal Processing Magazine, 40(1):98 114, 2023.
Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recognition. In Proc. IEEE Conf. on Compuler Vision and Pallern Recognition, pp. 770-775 770-778, 2016.
David P Hoffman, Isaac Slavitt, and Casey A Fitzpatrick. The promise and peril of deep learning microscopy. Nature Methods, 18(2):131-133 120811111,221111 2021.
Eliahu Horwitz and Yedid Hoshen. Conffusion: Confidence intervals for diffusion models. 2022. doi: 10.48550/ ar Kiv.2211.099
Mihir Joshi, Aaron Pruitt, Chong Chen, Yingmin Liu, and Rizwan Ahmad. Technical report random cartesian sampling for dynamic MRI. arXiv:2206. 03630, 2022.
Sergey Kastryulin, Jamil Zakirov, Nicola Pezzotti, and Dmitry Dylov. Image quality assessment for magnetic resonance imaging. IEEE Access, 11:14154 14168, 2023.
Michal Klein, Louis Bethune, Eugene Ndiaye, and Marco Cuturi. Multivariate conformal prediction using optimal transport. arXiv:2502. 03609, 2025.
Florian Knoll, Kerstin Hammernik, Chi Zhang, Steen Moeller, Thomas Pock, Daniel K Sodickson, and Mehmet Akcakaya. Deep-learning methods for parallel magnetic resonance imaging reconstruction: A survey of the current approaches, trends, and issues. IEEE Signal Processing Magazine, 37(1):128 140, January 2020.
Roger Koenker and Gilbert Bassott. Regression quantiles. Econometrica, 46(1), 1978. doi: 10.2307 09907/1913643.
Gilad Kutiel, Regev Cohen, Michael Elad, Daniel Freedman, and Ehud Rivlin. Conformal prediction masks: Visualizing uncertainty in medical imaging, In Proc. Intl. Conf. on Learning Representations, 2023.
Jing Lei and Larry Wasserman. Distribution free prediction bands for non-paramttric regression. Journal of the Royal Statistical Sociely, 76, 2014. doi: 10.1111 1.1111/ssbb.2021
Jing Lei, Max G'Sell, Alessandro Rinaldo, Ryan Tibshirani, and Larry Wasserman. Distribution-free predictive inference for regression. Journal of the American Statistical Association, 2018
Soundouss Messoudi, Sébastien Destercke, and Sylvain Rousseau. Conformal multi-target regression using neural networks. In Proc. Symp. on Conformal and Probabilistic Prediction With Applications, pp. 65-83.
PMLR, 2020.
Soundouss Messoudi, Sébastien Destereke, and Sylvain Rousscau. Copula- based conformal prediction for multi-target regression. Pattern Recognition, 120:108101, 2021.
21

---

Published in Transactions on Machine Learning Research (11/2025)
Soundouss Messoudi, Sébastien Destercke, and Sylvain Rousseau. Ellipsoidal conformal inference for multi- target regression. In Proc. Symp. on Conformal and Probahilistic Prediction With Applications, pp. 294-306. PMLR, 2022.
Matthew Muckley, Bruno Riemenschneider, Alireza Radmanesh, Sunwoo Kim, Geunu Jeong, Jingyu Ko, Yohan Jun, Hyungseob Shin, Dosik Hwang, Mahmoud Mostapha, al. Results of the 2020 fastMRI challenge for machine learning MR image reconstruction. IEEE Trans. on Medical Imaging, 40(9):2306 2317, 2021.
Dominik Narnhofer, Andreas Habring, Martin Holler, and Thomas Pock. Posterion-variiance--sed error quantification for inverse problems in imaging. SIAM Journal on Imaging Sciences, 17(1):301-333 2024.
Roger Nelsen. An Introduction to Copulas. Springer NY, 2nd edition, 2006.
Gregory Ongie, Ajil Jalal, Christopher A Metzler, Richard G Baraniuk, Alexandros G Dimakis, and Rebecca Willett. Deep learning techniques for inverse problems in imaging. IEEE Journal on Selected Areas in Information Theory, 1(1):39 56, May 2020.
Harris Papadoponlos, Kostas Procdrou, Volodya Vovk, and Alex Gammerman. Inductive confidence machines for regression. In Proc. European Conf. Machine Learning, pp. 345-356 2002. doi: 10. (0.1007/3-540-33655511 29.
Ji Won Park and Kyunghyun Cho. Semiparametric conformal prediction. In Proc. Int. Conf. on Artificial Intelligence and Statistics, pp. 1880-3888, 2025.
Yaniv Romano, Evan Patterson, and Emmanuel J. Candès. Conformalized quantile regression. Tn Proc. Neural Information Processing Systems Conf., pp. 3543 3553, 2019. doi: 10.48550/arxiv.1000 1905.03222
Aviv A. Rosenberg, Sanketh Vedula, Yaniv Romano, and Alex M. Bronstein. Fast nonlinear vector quantile regression. In Proc. Intl. Conf. on Learning Representations, 2023.
Max Sampson and Kung-Sik Chan. Conformal multi-target hyperrectangles. Statical Analysis and Data Mining, 17(5), 2024.
Swami Sankaranarayanan, Anastasios N. Angelopoulos, Stephen Bates, Yaniv Romano, and Phillip Isola. Semantic uncertainty intervals for disentangled latent spaces. In Proc. Neural Information Processing Systems Conf., 2022. doi: 10.48550 /115500///iv.222 220....0220.1000.4..
Amuroop Sriram, Jure Zbontar, Tullie Murrell, Aaron Defazio, C. Lawrence Zitnick, Nafissa Yakubova, Florian Knoll, and Patricia Johnson. End-to-end variational networks for accelerated MRI reconstruction. In Proc. Inil. Conf. on Medical Image Compulation and Compuler-Assisted Intervention, pp. 64 73, 2020.
Sophia Huiwen Sun and Rose Yu. Copula conformal prediction for multi-step time series prediction. In Proc. Intl. Conf. on Learning Representations, 2024.
Jacopo Teneggi, Matthew Tivnan, J. Webster Stayman, and Jeremias Sulam. How to trust your diffusion model: A convex optimization approach to conformal risk control, 2023.
Gauthier Thurin, Kimia Nadjahi, and Claire Boyer Optimal transport- based conformal prediction. In Proc. Intl. Conf. on Machine Learning, 2025.
Ryan J Tibshirani, Rina Foygel Barber, Emmanuel Candes, and Aaditya Ramdas. Conformal prediction under covariate shift. In Proc. Neural Information Processing Systems Conf-, volume 32, 2019.
Matthew Tivnan, Siycop Yoon, Zhennong Chen, Xiang Li, Dufan Wu, and Quanzheng Li. Hallucination index: An image quality metric for generative reconstruction models. In Proc. Intl, Conf. on Medical Image Computation and Computer-Assistern Intervention, pp. 449-458, 2024.
Sumcet Virmani, Rashmi Virmani, Jagadeesh Singh, and Amjad Ali. Imaging artifacts: A pictorial review. Journal of Nuclear Medicine, 56, May 2015.
22

---

Published in Transactions on Machine Learning Research (11/2025)
Vladimir Vovk, Alexander Gammerman, and Glenn Shafer. Springer, New York, 2005.
Algorithmic Learning in Random World.
Vladimir Vork, Jieli Shen, Valery Manokhin, and Min-ge Xie. Nonparametric predictive distributions based conformal prediction. In Conformal and probabilistic prediction and applications, pp. 82-102, 2017.
Zhou Wang. Applications of objective image quality assessment methods. TEEE Signal Processing Magazine, 28(6):137- 2011.
Zhou Wang, Alan C Bovik, Hamid R Sheikh, and Eero P Simoncelli. Image quality assessment: From error visibility to structural similarity. IEEE Trans. on Image Processing, 13(4):600 612, April 2004.
Jeffrey Wen, Rizwan Ahmad, and Philip Schniter. A conditional normalizing flow for accelerated multi-coil MR imaging. In Proc. Intl. Conf. on Machine Learning, 2023.
Jeffrey Wen, Rizwan Ahmad, and Philip Schniter. Task-driven uncertainty quantification in inverse problems via conformal prediction. In Proc. European Conf. on Computer Vision, 2024.
Jeffrey Wen, Rizwan Ahmad, and Philip Schniter. Conformal bounds on full-reference image quality for imaging inverse problems. Trans. on Machine Learning Research, 2025. ISSN 2835-8856.
Jure Zbontar, Florian Knoll, Amuroop Sriram, Matthew J. Muckley, Mary Bruno, Aaron Defazio, Marc Parente, Krzysztof J. Geras, Joe Katsnelson, Hersh Chandarana, Zizhao Zhang, Michal Drozdzal, Adriana Romero, Michael Rabbat, Pascal Vincent, James Pinkerton, Duo Wang, Nafissa Yakubova, Erich Owens, C. Lawrence Zitnick, Michael P. Recht, Daniel K. Sodickson, and Yvonne W. Lui. fastMRI: An open dataset and benchmarks for accelerated MRI. orXiv:1811.08883 2018.
Richard Zhang, Phillip Isola, Alexei A Efros, Eli Shechtman, and Oliver Wang. The unreasonable effectiveness of deep features as perceptual metric. In Proc. IEEE Conf. on Computer Vision and Pattern Recognition, pp. 586 595, 2018.
Ruiyang Zhao, Burhaneddin Yaman, Yuxin Zhang, Russell Stewart, Austin Dixon, Florian Knoll, Zhengnan Huang, Yvonne W. Lui, Michael S. Hansen, and Matthew P. Lungren. fastMRI+: Clinical pathology annotations for knee and brain fully sampled magnetic resonance imaging data. Scientific Data, 9(1):152, 2022. doi: 111110//-111777-2220255555
23

---

Published in Transactions on Machine Learning Research (11/2025)
A Proof of Theorem 2
In this section, show that A(dea) in 27 converges to X in (23 the definition of almost-sure convergence.
and Vluune oo. We first recall
Definition 1 (Almost-sure convergence). Let (Xn)n21 be sequence of random variables defined on probability space (.,F.P). We sny that Xa converges almost surely (or with probability 1) to random variable X, denoted as Xn
a.s. X, if
Pr lim Xn =x =1.
That is, the outcomes w e Q under which (w) converges X(w) occur with probability one.
We now state two theorems that form the basis for our convergence analysis.
Theorem 3 (Glivenko-Cantelii (Fristedt & Gray 2013
Suppose XI,...,Xx Xn are i.idd random variables
with CDF F()). Define the empirical CDF as
X1 S x,
F(x)
Then F()) converges uniformly to F() almost surely, i.e.
sup
[(xz) - F(x)] a.s. 0.
ZER
Theorem 4. Let X1, X2,..... ... Xn be iidd random variables with CDF F(.). Define the quantile al level 10 as
Q(p) innff:: F(x)) pP)
and the empirical quantäle at level E (0, 1) as
Qx(p) = inf(æ F(x) 2 p},
where F(-) is the empirical CDF, For fixed level p, construct the n- ne-ependent level
Tn [p(n+) +
n
which approaches p as n oc. If F()) is continuous and strictly increasing Q(p), then
Qn(m)) 0.2, Q(p).
That is, the empirical quantile at level Tn converges almost surely to the true quantile at level p.
Proof. First, we analyze the convergence of Tn. Observe that
Tn [p(n + 1)1 p(n 1) An
=p P+ An
(44)
where A, 1) accounts for the rounding of the ceiling function. Thus
2
p< Jn <pt
(45)
and lim Th p. Next, look to bound (m()) - co. For any fixed O, and assuming F(.) is continuous and strictly increasing at Q(v), we have
F(Q(p) - +
24

---

Published in Transactions on Machine Learning Research (11/2025)
And by Theorem the Glivenko- Clivenko-Cantelli theorem, F(-) converges uniformly to F(+) almost surely. This means that, for any > 0, there almost surely exists an N such that, for all n 2 N and for all IR,
[F(z) F(2) S 8.
By choosing
min F(Q(p) - c), F(Q()) + -
we get
< F(Q()) + 6,
(46)
and so
F(Q(p) c) < F(Q(p))
(47)
for all n 2 N. We now establish two intermediate results.
Lemma 5. For sufficiently large n, we have Qn(mm) 2 Q(p) -.
Proof. We prove the claim using contradiction. Suppose that Qn(m)) < Q(p) Then, due the non-dccreasing property of F(.), we have
F(Qn(T)) F(Q(p) c)
(48)
for any n. Furthermore, since F(Qn(1m) 2 Yn for any n by the definition of the empirical quantile, and since N/ > p from 450, we have
F(Q(P) c) Ta p. when n 2 N. This implies that Qn(%)) 2 Qp)) - for sufficiently large n.
(49)
However,
49
contradicts
47
Lemma 6. For sufficiently large n, we have Qa(m)) S Q(p) +c.
Proof. We prove the claim using contradiction. Suppose that Qn(mm) > Q(p) + c. Recall that, by definition, Qm(mm) inf(x: F(x) 2 }n). Thus if Q(p) +e Qu(Ta) then
F(Q(p) +e) < <mn-
(50)
And recall from 46 that F(Q(p) +c) 8 p, equivalently that
5 F(Q(p) +
> 2
(51)
From Theorem 3 the Glivenko Cantellli thoorem, F-)) converges uniformly F()) almost surely. This means that, for the given 6, there almost. surely exists N' such that, for all 2 N' and any R
5
8
[x() F(x)
F(x) F(x) 2
(52)
2
Combining 51 and
12
we have that, for all n 2 N'
8 > p+t
8
F(Q(p) + + 2 F(Q()) ()
+
2
2
From 45
that, for 2 4/5,
8
7/ < p < 2
Thus, for sufficiently large n, we have
f(Q(p)) + c) > 7/o
which contradicts
50
This implies that Qx(mn) 5 ((p) + for large
25

---

Published in Transactions on Machine Learning Research (11/2025)
Lemma 5 and Lemma 6 hold almost surely for an arbitrary 0, and together say that
Q(p) < - Qn(%n) & {p() +
for sufficiently large n. Since make arbitrarily small, have that
lim Qa(mn) = Q(p),
n-000
almost surely, and thus (m() a. Q(p).
Having established Theorem 3 and Theorem we now return to our main objective, which is proving that the X(deal) in 27 converges to the À in (33 For clarity, we restate Theorem here.
Theorem (Restatement of Theorem
For each target component k = I,...,,, K, suppose that the non- conformity scores {St, 17- are i.i.d with CDF (-,()), and for - maxy, Fs. (Sk), suppose that Fr(.) is continuous and strictly increasing at the (1-a)-level quantile T. T. Then M(dea) from (27) converges to X
from 23 almost surely as n 00 and Nuuuee + 20.
Proof. We first analyze the effect of Nuune + oo for an arbitrary fixed n. Recall that the empirical CDF Fs. (-) of the nonconformity score for the kth component is computed as in (24g using the luning samples {Si, Matter. From Theorem 3 Fs, (+) converges uniformly to the CDF Fs, (.) almost surely Tune - As a result, it follows that for each calibration nonconformity score Si,,, where E fl...., n., we have
Sikk A Fs. (St,k) a.S. Fs, (Si.))
Let us now consider the maximum transformed score 5, max (3 ) defined in (26). Since the maximum function is contimuous everywhere on K and Fs, (S,.) a.s. Fs, (St,)), the continuous mapping theorem implies that
88 Thune + X0, recalling the definition of the transformed score Sr,x from 25
S, max Fs, (S,,a)
a.s. max Fs, (St,k) & Ti
as Atuune + 00. Becanse {S 1:=1 are assumed to be i.id with CDF Fs. (-), we see that (Ti) are i.dd with CDF F.(.) for T max; Fs,, (Sx).
Next, we analyze the effect of n 00. Let us denote the n-sample empirical quantile of T as Qn(-) and the quantile of T as Q(-). Recall from 27 that
\DDaa Qa [(1 a)
(I(1-a)n++ 111)
n
Because Fr(() is assumed to be continuous and strictly increasing at Q(1 - o)), Theorem establishes that, aS πI - 00,
[[1 (I +11) a.a, Q(1 a).
A(Dess) = Qn
(53)
n
Finally, recall the definition of from 23
X arg min s.t. Pr(nk Fs, (8A) 21/211
The constraint can be rewrilien as
Pr(max (Sx) S A = Pr(T <
k
which allows 23 to be rewritten as
X = arg mign s.t. Pr(T < to 2 a-a
(54)
26

---

Published in Transactions on Machine Learning Research (11/2025)
Table 3: Number of positive samples in the susp--sed subset of the fastMRI+ knee dataset.
Label
Positive Training Samples
Positive Validation Samples
Meniscus Tear
1921
335
Cartilage Partial Thickness loss/ /efectt
871
176
Joint Effusion
225
41
Bone-Fracture /ontusion/ disiioatiiiooo
97
6
Bone Subchondral edema
76
21
Table 4: Classifier performance the fastMRT+ validation set.
Label
Accuracy
Precision
Recall
AUROC
Meniscus Tear
0.6595
0.3005
0.9784
0.889
Cartilage Partial Thickness
0.6184
0.1558
0.8988
0.8564
Joint Effusion
0.9031
0.1356
0.8000
0.9465
Bone- Feane-racttue// Contusion Contusion/(iisloatioon /dislocation
Crracture/Contusinn
0.7715
0.0060
0.5000
0.7971
Bone Subchondral edema
0.5704
0.0127
0.5714
0.6338
Average
0.7046
0.1221
0.7497
0.8246
But the X in 54 is simply the (1 - ar-level quantile In other words,
= inf FT(A) a} Q(1 aa).
(55)
Finally, combining 53 with 55
conclude that
A(Deal) a.S. ā
a8 Vitune > oc and n >> ac.
B Classifier Details
We train the multi-label classifier on the K = labels with the most annotations in the non-fat-suppreesed subset of the fastMRI+ knee data from Zhho et al.] (2022). Table 3 shows the number positive samples for each those labels. Note that images with multiple instances of the same pathology only count as a single positive sample.
We implement and train the multi-label classifier using nearly the same procedure Wen el. al. 2024 In particular, we start by initializing standard ResNet-50 (He et al. 2016) with the pretrained ImageNet weights from (Deng et al. 2209)), after which we reduce the number of final-layer outputs to K = 5. Then we pretrain the network in a self-supervised fashion using the (unlabeled) non-fal-suppressed fastMRI knee data following the SimCLR procedure from Chen et al. (2020) with a learning rate 0.0002, batch size of 128, and 500 epochs. Finally, we perform supervised fine- linettunigg using binary cross-entropy loss on the fRst data, where we address class imbalance by weighting the loss contribution from each class by the ratio of negative labels to positive labels for that particular class. To encourage adversarial robustness, we the same l bg-boudded gradient ascent attack as Wen et al. (2024). and we train the classifier for 150 epochs with batch size of 128, learning rate of 5c-5, and weight decay of lc-7. Finally, we save the model checkpoint with the lowest validation loss. Performance on the validation dataset is shown in Table 4
27


Reviews:
[{"review_id": "snWxMHTK1G", "reviewer": "Reviewer_VTCb", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "**Strengths**\n- Uncertainty quantification for imaging inverse problems is an important topic\n- Multivariate CP is an open problem\n- Interval length minimization is central to CP methods\n\n**Weaknesses**\n- Presentation of the proposed method could be expanded on\n- Comparison with existing copula-based methods should be included given the similarity of the approach\n- Lack of finite-sample guarantees of the proposed method\n\nI will expand on these points below and I am looking forward to discussing with the authors!", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "Yes", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "The submission studies conformal prediction (CP) in imaging inverse problems. In particular, it focuses on joint marginal coverage of multiple quality or task-related metrics (e.g., PSNR, SSIM, post-softmax outputs of a classifier). The submission proposes an optimization formulation of multi-target CP, based on the intuition that coverage is monotonically non-decreasing with the size of the prediction sets. A finite-sample procedure is designed, which retrieves the solution of the optimization problem asymptotically.\n\nExperiments are performed on fast MRI data, for both reconstruction and downstream pathology classification. Results show that the proposed method provides coverage with generally tighter uncertainty intervals.", "claims_and_evidence_yn": "Yes", "claims_explanation": "", "audience_yn": "Yes", "audience_explanation": "", "requested_changes": "**Connections with Copula-based methods**\nCould the authors expand on the relation of the proposed method with existing copula-based CP alternatives? The optimization formulation in Eq. (20) is very similar to Eq. (8) in Sun et al., \"Copula-based CP for Multi-step Time Series Forecasting\" (2022).\n\nFurthermore, it is stated in Sec. 2.3 that \"there is no guarantee of satisfying a coverage guarantee like (9) with a finite calibration set\". This claim is true of Messoudi et al. (2021), who do not provide a finite-sample theorem. However, Sun et al. do show how to calibrate the copula estimation with finite samples (see Theorem 4.1). In particular, Conformal Predictive Distributions do allow to obtain finite-sample guarantees on estimates of the marginal empirical CDFs (see \"Nonparametric predictive distributions based on conformal prediction\", Vovk et al. (2017)).\n\nIt is important to compare the performance of the proposed approach with existing copula-based methods.\n\n**Motivation of proposed method**\nIf the core contribution of the submission is the minimax formulation, this should be emphasized and motivated better. Currently, the method is motivated by the observation that coverage is monotonically non-decreasing with the size of the prediction sets, which feels somewhat reductive. I think there are deeper, important considerations to discuss. For example, what happens with the current formulation if each \"target\" represents a different demographic? What if different FRIQ metrics have different importance for the downstream task at hand?\n\n**Finite-sample guarantees**\nAs mentioned above, I think there might exist finite-sample algorithms that would provide coverage. In its current formulation, Theorem 1 falls short of delivering on the promise of the method.\n\n**Multi-round measurement acquisition**\nCould the authors expand on the observation that \"we really desire that the multi-round coverage is at least $1 - \\alpha$\"? In imaging, for example, why do we want coverage to hold at each step? Intuitively, we want coverage to hold when we make a decision, i.e. at the end? Also why is multi-round coverage expressed as the sum of the single-round coverages? Again, this feels very close in spirit to Sun et al., where they also have a discrete time horizon, and the goal is to provide uniform coverage over time steps.\n\n**A few relevant works on multivariate conformal prediction**\nIt might be useful to include and discuss these citations to better place the contributions of the current submission within the broader CP research subfield:\n\n[1] \"Multivariate Conformal Prediction using Optimal Transport\", Klein et al. (2025)\n\n[2] \"Optimal Transport-based Conformal Prediction\", Thurin et al. (2025)\n\n[3] \"Multivariate Conformal Prediction via Conformalized Gaussian Scoring\", Braun et al. (2025)\n\n[4] \"Minimum Volume Conformal Sets for Multivariate Regression\", Braun et al. (2025)\n\n[5] \"Length Optimization in Conformal Prediction\", Kiyani et al. (2024)\n\n[6] \"Fast Nonlinear Vector Quantile Regression\", Rosenberg et al. (2023)\n\n---\n\n**Experiments**\n\nIn models, both an \"image recovery model $f$\" and a posterior sampling method $g$ are mentioned. Could the authors expand on how each is used for each calibration method? Do all method have access to the same predictions, or are there differences that could confound the results?\n\nIn Sec. 5.2, it is stated that \"the goal is to ascertain the presence/absence of each of $K=5$ different pathologies\", which would imply hard binary labels. Later, it is mentioned \"the true soft-output $z_{0,k}$\". Could the authors expand on where the soft-output ground-truth labels come from?\n\nCould the authors expand on the SC method and why it fails to provide coverage?\n\nIn the multi-round experiment, how are the thresholds $\\tau = 0.16$ and $\\tau = 0.10$ chosen? In the current form, these feel arbitrary, and might introduce bias in the results. Have the authors considered plotting mean interval length as a function of acceleration? Is there any guarantee that the interval length will be decreasing with the number of measurements?\n\n---\n\n**Minor comments**\n- \"and radiologists receive ...\". This sentence feels a little out of context, maybe introduce with \"For example\"?\n- \"only a single target\", \"a scalar target\" would be more precise\n- \"with certain statistical guarantees\", it is unclear whether this sentence refers to \"uncertainty intervals\" or \"black-box predictor\".\n- \"only the conformalized quantile regression (CQR) method of Romano et al.\". I am not sure I follow how Eq. (6) is the same as CQR. In CQR, one trains a predictor with the pinball loss, and subsequently calibrate such predictor. Here, calibrated quantiles are taken from empirical samples, without training a predictor. Furthermore, CQR uses a multiplicative parametrization of the prediction sets, but Eq. (7) uses an additive parametrization.\n- DISTS is not defined.\n- In Fig. 3, the labels of the axes may be misinterpreted by the subcaptions. I would suggest removing the subcaptions, since the information is already presented in the y-axis labels.\n- \"they are less interpretable, since the uncertainty intervals ... the other target components\". This claim is task-dependent. Sometimes, the dependence structure across different metrics is fundamental to obtain interpretable claims. In imaging, for example, principal components uncover important spatial relations.", "broader_impact_concerns": "No concerns", "additional_comments": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "cb1yrpqdmu", "reviewer": "Reviewer_Fe5k", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "Strengths:\n- The paper offers a theoretically-justified and experimentally effective strategy for obtaining multidimensional intervals with joint coverage\n- They avoid making significant assumptions like task independence and provide a general framework for combining together a wide variety of tasks\n-Their minimax formulation is well motivated in preventing severe coverage imbalance while simultaneously allowing for a simple and tractable algorithm\n-Their proof in demonstrating convergence in infinite samples also helps justify the algorithm as principled.\n\nWeaknesses/Clarifications (these will be further specified in the changes section):\n- In general, the evaluation of the method is fairly barebones and could be improved through additional experiments\n- The paper could use more comprehensive evaluation, especially in stratifying metrics across relevant groups (e.g., tasks, interval sizes)\n- The paper could include additional discussion on things like task balance, the implications of using a single parameter to control multiple coverages, and the consequence induced by using sample statistics in multiple areas (sample CDF, posterior sampler, etc). A discussion on tuning set size and it's effect is also warranted.\n-Another example is deploying more interesting settings like simultaneously covering a metric and downstream task.", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "Yes", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "The authors present a novel way to apply conformal prediction for obtaining multidimensional prediction interval that, on average, cover the true values of multiple predictions concurrently. They develop and apply this technique specifically in the context of downstream tasks on image reconstructions. Specifically, for MRI acceleration reconstructions, they show that their method generally produces smaller intervals than existing methods while still achieving joint coverage. \n\nThe heart of their technique is a minimax optimization formulation in which the task with the largest coverage is minimized while still ensuring joint coverage. They develop an algorithm that approximates a solution to this optimization and correct it to produce valid intervals in finite samples using split conformal prediction. They also prove that the algorithm produces the true solution given infinite samples. They then apply this technique for 3 different experiments: multi-metric coverage, multi-task coverage, and multi-round acquisition. \n\nTo validate their method, they run each of these experiments in the context of upsampling undersampled MRIs. They train and test a reconstruction network, posterior sampler, and classifier on the fastMRI dataset. They show that they can achieve joint coverage while also minimizing the average set size, improving upon previous techniques.", "claims_and_evidence_yn": "Yes", "claims_explanation": "", "audience_yn": "Yes", "audience_explanation": "", "requested_changes": "Major concerns (needed for acceptance)\n- I believe the authors should report per-target marginal coverage. That is, report the coverage stratified by each task/metric separately.\n    - Though the minimax formulation helps prevent coverage imbalance between the targets, it can still be the case that  one target could still achieve a much higher coverage than others, especially because there is only a single conformal threshold over all targets. The paper does not report marginal coverage per target, so it's difficult to assess whether this occurs in practice. Since metrics/tasks can vary significantly in difficulty, this could be a very real possibility. Indeed, the classification performance in the MRI task example varies widely due to the class imbalance issue.\n- I’m also a bit concerned with the reliance on posterior sampling. Many of the methods presented here require access to posterior samples, which may or may not be accurate and could also be difficult to obtain. This merits a short discussion about the implications of bad posterior samples and alternative strategies if such a sampler is not attainable (e.g., directly doing quantile regression with the task networks).\n\nSuggestions (not needed for acceptance but would strengthen)\n - The notation can be a bit verbose at time. I think certain things like defining h_k may be unnecessary since the predictions /hat{z} are already defined and shown as the output of task /mu_k or metric m.\n - All the subsections of section 4 contain very similar language (like the last sentence of each subsection). This can be tightened and condensed.\n - Details about the performance of the metric estimation task would also be valuable, similar to what was done for the task case. That way trends about the coverage performance/set sizes can be correlated with prediction accuracy.\n - I think a more challenging experiment would be valuable to display. For example, covering a metric and task simultaneously is potentially challenging because of the difference in scales and the complex dependency between them but should be something this method could handle. It would also be of great value to understand to what degree making good quality reconstructions affects classification accuracy. Especially if you also trained a network that directly classified from reconstruction images instead of clean images.\n\nMinor Clarifications \n- In section 4.3, what are the meainging of Z_1, Z_2, Z_3,…(which you assume are exchangeable) My understanding is that the goal is to predict a single Z_0 through the procedure? Are these the calibration/tuning data points?\n- Why use a classifier that is trained on clean images, shouldn’t you use one trained on the images from the reconstruction methods? \n- Why is your method worse for the meniscus tear classification? \n    - Is this at all correlated with the class imbalance (meniscus tears are overrepresented)", "broader_impact_concerns": "As a minor concern, as often comes up with conformal prediction work, I think it is imperative to address the limitations of the guarantees that conformal provides. Though having rigorous uncertainty quantification is extremely positive for scientists, it is also critical for them to understand the nature of the prediction intervals (i.e., marginal vs conditional coverage) and how it applies to their specific context. To that end, it may be worth including a small discussion in the broader impacts statements that gives an example of how exactly an MRI scientists should interpret and utilize the results given in this manuscript.", "additional_comments": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "nnWo6p6O2e", "reviewer": "Reviewer_kgLm", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "Strengths:\n\n+ The proposed method, as showcased in the paper, obtains more robust/better performance than existing conformal-type methods in the field of medical imaging, according to several success metrics (including a domain specific one --- namely better achieved acceleration in MRI).\n\n+ The proposal is a natural one from the minimax perspective, and a simple one --- it obtains a bounding box in higher dimensions, foregoing more sophisticated and potentially less robust/interpretable approaches with different geometries (ellipsoidal etc.). (More contrived prior approaches include the Sampson and Chan (2024) method with an extra quantile-regression-based score renormalization step, which this paper includes in the discussion/evals.)\n\nWeaknesses:\n\n- While the proposed approach is well-situated by the authors in the context of conformal prediction based tools, that is not so with respect to the slightly more expansive toolkit of distribution-free UQ. In particular, in the multi-target context, the well-known Learn then Test (LTT) method of Angelopoulos et al pioneered a hypothesis-testing framework that helps obtain guarantees that can simultaneously bound multiple notions of (monotonic or nonmonotonic) risk (not restricted to, but including, miscoverage rates). LTT is also in particular well-established in the empirical terrain, including CV applications. Thus, this appears to be a substantial omission.\n\n- The empirics, while conducted in a (from what I can tell) well-established and motivated medical imaging setting, lack a certain amount of details. For instance, the Sampson and Chan QN method, that is directly compared to, aims to combine the scores into one (similar to the presented method), but they go through the more convoluted step of renormalizing the scores using quantile regression. Their point is to mitigate the disparity between the different dimensions' scores' magnitudes, which at first glance their formula manages to achieve in some sense. However, the minimax approach indeed achieves (at least somewhat) better performance in presented experiments, which makes one wonder: Is this due to the QN method being unable to handle the differing scale or distribution of the different metrics' values?\n \n    ---> For this and other typical multi-objective reasons, I would have liked to see more visualizations of the different score dimensions, to help the reader better understand the trade-offs that the tested methods have to navigate. \n\n    ---> In a similar vein, I would have liked to see some well-thought-out synthetic experiments on the proposed method comparing it to prior approaches, to better understand: to what extent is the proposed method better than the others due to some special structure in medical imaging (inverse) tasks, and to what extent would that advantage erode under different relative distributions of the different dimensions' scores? In particular, the method doesn't directly control the interval length, and therefore it wouldn't necessarily always outperform the other methods in that metric.\n\n- The manuscript's branding as minimax, to my mind, necessitates further exploration of the following questions. \n    ---> First, the natural conceptual/qualitative question that this manuscript entails but elides is thus: how is it the case that a minimax (and therefore theoretically conservative) approach outperforms other methods that may not have such conservatism explicitly built-in? \n   ---> Second, the method is to aggregate the scores by taking their max. Instead, if taking a soft-max (according to some surrogate soft-max), would that help \"smooth out\" the method in desirable ways?\n   ---> Also, as a minor qualm about the paper positioning itself in the context of minimax optimization: E.g. when the intro speaks about the proposed method as \"based on minimax optimization\", that is a bit confusing/could feel misleading to some; while it's certainly the case that the method solves a minimax optimization problem, it does not actually need to use any of the machinery contained in the minimax optimization literature, due to the very simple structure of the problem in this case. Thus, it would be more sensible to e.g. frame the contribution as stemming from \"a minimax formulation\", or \"a minimax intuition\".", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "Yes", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "The paper presents a new conformal prediction method for multi-target settings, which ensures simultaneously valid prediction sets for one or more real-valued targets, and tests this method in the context of accelerated MRI experiments. The method stems from a minimax formulation, which essentially finds the best prediction interval parameter lambda out of those that ensure joint distribution free coverage in every dimension of the target vector. Expected validity guarantees, including provable coverage asymptotically and in finite samples, are provided under standard assumptions. Aside from single-shot settings (such as measuring multiple metrics on a recovered image), this method is also applied to provide coverage in the context of multi-round \"measurement acquisition\" schemes. The experiments showcase the improvements in metrics such as coverage rates and prediction interval length in each dimension of the target.", "claims_and_evidence_yn": "Yes", "claims_explanation": "", "audience_yn": "Yes", "audience_explanation": "", "requested_changes": "As per the above, here are the main questions that I consider important to address:\n\n1. Please introduce a detailed discussion comparing to the Learn-then-Test approach and resulting methods based on multiple hypothesis testing rather than the conformal approach. In particular, I'd like for such a comparison to happen from the (a) methodological/expository point of view (i.e. a careful discussion of how LTT achieves its guarantees vs. the proposed method, and in particular a discussion for the readers about the difference in provided guarantees), and (b) experimental point of view (i.e. does the proposed method achieve less conservative coverage/length than LTT?).\n\n2. Empirically, I would like to request extra visualizations that would (a) better showcase the tradeoffs induced by the multiple objectives, to better understand how the different methods that are compared deal with them; and (b) compare the methods on another task, preferably a well-designed synthetic one, in particular to better delineate the limitations of the proposed method due to its conservatism.\n\n3. (Less pressing than above two, but still quite desirable) A more focused discussion of the minimax nature of the method as per the last of the Weaknesses; this would help the readers navigate (a) the interesting contrast between a robust nature of the proposed method vs. its more efficient performance than the alternatives; and (b) understand future work avenues, such as those stemming from replacing max -> softmax.", "broader_impact_concerns": "N/A", "additional_comments": "", "extra_scores": {}, "extra_text": {}}]

Author rebuttal:

Decision: Accept (poster)