Conformal Calibration of Statistical Confidence Sets
Authors: Luben Miguel Cruz Cabezas
Guilherme Soares
Thiago Ramos
Rafael Bassi Stern
Rafael Izbicki
Venue: Accepted by TMLR ()

Abstract: Constructing valid confidence sets is a crucial task in statistical inference, yet traditional methods often face challenges when dealing with complex models or limited observed sample sizes. These challenges are frequently encountered in modern applications, such as Likelihood-Free Inference (LFI). In these settings, confidence sets may fail to maintain a confidence level close to the nominal value. In this paper, we introduce two novel methods, TRUST and TRUST++, for calibrating confidence sets to achieve distribution-free conditional coverage. These methods rely entirely on simulated data from the statistical model to perform calibration. Leveraging insights from conformal prediction techniques adapted to the statistical inference context, our methods ensure both finite-sample local coverage and asymptotic conditional coverage as the number of simulations increases, even if the observed (real) sample size n is small. They effectively handle nuisance parameters and provide computationally efficient uncertainty quantification for the estimated confidence sets. This allows users to assess whether additional simulations are necessary for robust inference. Through theoretical analysis and experiments on models with tractable and intractable likelihoods, we demonstrate that our methods outperform existing approaches, particularly in small-sample regimes. This work bridges the gap between conformal prediction and statistical inference, offering practical tools for constructing valid confidence sets in complex models.

Published in Transactions on Machine Learning Research (04/2026)
Conformal Calibration of Statistical Confidence Sets
Luben M. C. Cabezas
wttp:////...bbammal...cmoo com
Department of Statistics and Institute of Mathematics and Computer Science Federal University of São Carlos and University of São Paulo São Carlos, SP 18565-905 and 18566-590, Brozil Université Grenoble Alpes, Inria, CNRS, Grenoble INP, LJK Grenoble, France
Guilherme P. Soares*
gwilherme.soures.222 25 uk
Department of Statistical Science University College London London, WC1E 6BT, UK
Thiago R. Ramos
thiagorr br
Department of Statistics Federal University of São Carlos
São Carlos, SP 18565-905, Brozil
Rafael B. Stern
rbstern@gmaiil.cmm com
Institute of Mathematics and Statistics University of São Paulo
São Paulo, SP 05308-090, Bruzil
Rafael Izbicki
mafuelizbicki@@maill.co com
Department of Statistics Federal Universily of São Carlos
São Carlos, SP 13565-905, Brazil
Reviewed on OpenReview: https: // openreview. net/ forum? id- J41X22VVE6
Abstract
Constructing valid confidence sets crucial task statistical inference, yet traditional methods often face challenges when dealing with complex models or limited observed sam- plc sizes. These challenges are frequently encountered in modern applications, such as Likelihood-Free Inference (LFI). In these settings, confidence sets may fail to maintain confidence level close to the nominal value. In this paper, we introduce two novel meth- ods, TRUST and TRUST++, for calibrating confidence sets to achieve distribution-fred condi- tional coverage. These methods rely entirely on simulated data from the statistical model to perform calibration. Leveraging insights from conformal prediction techniques adapted to the statistical inference context, our methods ensure both finite-sample local coverage and asymptotic conditional coverage as the number of simulations increases, even if the observed (real) sample size is small. They effectively handle nuisance parameters and pro- vide computationally efficient uncertainty quantification for the estimated confidence sets. This allows users to assess whether additional simulations are necessary for robust inference. Through theoretical analysis and experiments on models with tractable and intractable like- lihoods, we demonstrate that our methods outperform existing approaches, particularly in small-sample regimes. This work bridges the gap betwoen conformal prediction and sta- listical inference, offering practical tools for constructing valid confidence sets in complex models.
Work done while student the Institute Mathematics and Computer Science, University São Paulo.

---

Published in Transactions on Machine Learning Research (04/2026)
1 Introduction
Confidence sets are fundamental tools in statistical inference, allowing rescarchers to constrain the value of a parameter 0 E θ based on observed data X E X. A confidence set R(x) C θ is considered valid from a frequentist perspective if it satisfies the condition
P E R(X) = a 4A E
(1)
where a E (0, 1) is a predefined significance level. This mcans that R must achiove the correct coverage regardless of the true value of 0.
Traditional methods for constructing confidence sets, such as those detailed in standard textbooks (DeGroot & Schervish 2012 Schervish 2012 Casella & Berger 2024), are often inadequate when applicd to complex modern models. For inslance, construcling confidence sets lor mixlure models is notably challenging because standard asymptotic results may not hold (Chen & L, 2009 Wichitchan et al. 2019). Additionally, many traditional methods rely on asymptotic distributions, making them unsuitable for problems with small sample sizes (n).
The challenges bocome more difficult in Likelihood- Inference (LFT) scenarios, where the statistical model is implicitly defined by a complex simulator of X and the likelihood function is intractable (Izbicki et al.,
2014 Lueckmann al. 2019 Izbicki et al. 2019 Papamakarios al. 2019 Cranmer et al. 2020). In such
cases, test statistics must be estimated, and result, traditional methods for constructing confidence sets often perform poorly (Dalmasso ct al. 2024
This work aims to calibrate confidence sets to achieve conditional coverage (Equation D, even for challenging models. We focus on confidence sets defined as:
R(X) =
Ca),
(2)
where the cutoff Cp chosen to ensure conditional coverage, and T measures the plausibility that was generated from (in the language of hypothesis tests, is test statistic). We assume 7 is fixed (see Sections 1.1.1 and 1.1.2 for details on its selection) and provide tools for calibrating Cg.
Our calibration process only assumes the availability of simulated dataset of independent pairs, ((1,,,, Xi),..))..... (On, XR)}, where each 0, is drawn from some reference distribution r(0) and each X, is generated from the statistical model with parameters 0,, i.e., X ~ X 0s. We assume that r(X, 0) is strictly contimuuus for every 9. Note that each X,, represents the entire dataset that may be observed, rather than a single sample point.
Novelty. While several methods have been proposed to calibrate Cg for general problems (Section I.1 our method offers several new contributions:
1. Distribution-free guarantees with B-asymptotic conditional coverage. Our approach con- structs confidence sets (X): == E +(X, 0) 2 Co) with finite-sample local coverage:
P R E A) = 1 1-a, a,
where A is a subset of θ chosen so that
P R( E P 1P(8E Rn
Additionally, our method provides B-asymptotic conditional coverage:
lim TP (P R = a.
B Boc
Thus, our method robust to poor estimates of Co, offering distribution-fre guarantees while maintaining B-asymptotie validity. Notably, this coverage not asymptotic with respect to the size
2

---

Published in Transactions on Machine Learning Research (04/2026)
of the observed dataset x, n, is common with traditional asymptotic approximations. Instead, it relies solely on the number of simulations, B, which can generally be increased given sufficient computational resources. Furthermore, the set A is chosen to ensure that local coverage approximates conditional coverage.
2. Uncertainty quantification. In most cases, except for rare instances where Cs can be computed directly (Section 1.1,, current methods provide only an estimate Co of the true Cg. As result, the confidence intervals produced are only approximations of the true intervals which inherently contain errors. However, these methods do not offer way to quantify the uncertainty surrounding these estimated intervals. In contrast, our method explicitly constructs confidence sets that provide reliable uncertainty quantification around the estimated confidence intervals. Furthermore, this uncertainty quantification is computationally efficient and does not rely on costly procedures such as bootstrapping. Moreover, it allows users to assess whether additional simulations necessary for robust inference.
3. Effective handling of nuisance parameters. Most models involve nuisance parameters, which are not of direct interest but can complicate inference. Current methods often struggle with prop- orly handling nuisance parameters, particularly when they are numcrous, as they typically require expensive numerical maximization algorithms in high-dimensional spaces to ensure coverage (Dal- masso al., 2024). In contrast, our method significantly reduces the computational burden by transforming the high-dimensional maximization problem into one of finding the maximum over small, finite set of points, allowing for much more efficient optimization.
Figure compares TRUST+ with other methods for constructing confidence intervals for the slope coefficient in the generalized linear model explored in Scction 5.3 Despite the presence of 3 muisance paramcters, TRUST++ produces intervals that are closer to the oracle confidence set, both in terms of size and empirical coverage. It even outperfomm traditional GLM confidence sets based on x² approximations, as well as Monte Carlo-based intervals.
Our tools are developed using insights from conformal prediction techniques (sce Section 1.1 for related work). However, while conformal methods focus on generating prediction sets for new labels Y, which are random variables, statistical inference aims to create confidence sets for a fixed parameter that generated the data x. Thus, conformal techniques need to adapted to this purpose. In particular, asymptolic conditional coverage, which essential for valid sets, not guaranteed by standard conformal methods.
We provide two methods:
TRUST, which uses regression trees to offer a fast approximation to Ca.
TRUST++, which enhances TRUST by employing bagging of trees to deliver more accurate estimates. The exlension from trees to forests is non-trivial due to the need for finite sample guarantees.
The remaining of this work is organized as follows: Section contains discussion on related work. Section details the methodology, including the proposed methods and estimation procedures. Section presents the theoretical results. Section 4 discusses implementation details. Section provides applications to examples with tractable likelihood functions, likelihood-free inference, and nuisance parameter problems. Section 0 offers final remarks. Additional materials are provided in Appendices A through C including experimental details, supplementary algorithms, and complete thcorotical proofs. Finally, all code and supplementary materials are available in the GitHub repository: Ihttps://githuu http://gthhuub.om.o (Moooo//oooooooS5S5
1.1 Relation to Other Work
1.1.1 Classical Confidence Sets
Confidence sets typically follow the form described in Equation 2
The function =(X,6) can be a pivotal
quantity, which is quantity whose distribution does not depend on 0 (e-g., a --1 n1) when X ~ N(u,o)
3

---

Published in Transactions on Machine Learning Research (04/2026)
Inside
Undeterminate
A1
81 =
Grade
Boasting
MC
Asymptotic
TRUST+
TRUST+ uncertainty
-0.5
0.0
0.5
1.0
1.5
2,0
-0.5
0.0
0.5
1.0
1.5
2.0
B1
(a) 95% confidence intervals. Blue dotted lines represent lower and upper bounds for the oracle confidence interval.
B:
B
1.0%
Deviance from oracle coverage
0.0%
0.0%
-1.0%
-1.0%
-2.0%
22.0
-3.0%
-3.0%
-5.0%
-4.0%
Asymplotic
MC
Boosting
TRUST+
Asymptotic
MC
Boosting
TRUST++
Methods
Methods
(b) Deviance from oracle coverage.
Figure Confidence intervals and deviance from oracle coverage comparisons for the GLM example with sample size of 50. Our method closcly approximates the oracle in terms of both the estimated confidence intervals (top) and coverage probabilities (bottom), while also providing uncertainty quantification for the intervals. This feature highlights regions where the true parameter value is more likely to lie. Tf the undetermined region is large, users can increase the number of simulations to reduce its size and refine the results.
and a is known). Another example of 7 r(X,0) 0) is test statistics used for testing the null hypothesis that the data was generated from 8, such as the likelihood ratio statistic fxx(f) supree f(x(0") Lehmann Romano 2005).
A key challenge is computing Cg. In simple cases, Ca can be determined in closed form. For instance, when is pivotal quantity, Co does not depend 0 and is typically straightforward to find (Casella & Berger 2024). More often, asymptotic derivations are employed, which are effective only for large sample sizes n and for certain test statistics (Van der Vaart 2000). However, these approxinations require regularity conditions that frequently fail Algeri et al., 2019)- Similarly, boolstrap methods can be used, but these are effective only for large sample sizes n and also require regularity conditions [Van der Vaart 2000).
Our methods instead provide consistent (as increases) estimates of Co for any fixed т, even for small sample sizes We only require simulated dataset. ((1,, (1), (0n, Xn)).

---

Published in Transactions on Machine Learning Research (04/2026)
1.1.2 Likelihood-Free Inference
Likelihood-Free Inference (LFI) problems are problems in which the statistical model lack a tractable an- alytical distribution for X 0, but it is still possible to generate simulations from it. For comprchensive review, see Cranmer et al. (2020). In such situations, needs to be defined from the simulated set as well. For instance, one may use 7(x. 0b) as [ - 2 where the conditional mean and variance are estimated using the simulated datasct (Masscrano et al.) 2023 For other possibilitics, see c.g. Brehmer
et al. 2020;; Dalmasso el. al. [0024] Carzon et al. [2025b].
In LFI, once T has been defined, C3 is often estimated via Monte Carlo simulations for each fixed 0, but this approach becomes prohibitively expensive in higher-dimensional spaces. Alternatively, Dalmasso et al. (2020) 2024); Masserano el al. (2023) observed that Co is the 1 a conditional quantile of r (X, 0) on 0, and thus used quantile regressors to estimate Co using the simulated dataset {(11, XI)...., (0g, XB)}, where A is drawn from a reference distribution r(0) and Xs is drawn from the statistical model at Og. These methods recover Co B - oo, but they do not offer finite-sample guarantees. Complementary work has studied diagnostic tools for conditional density models and Bayesian inference algorithms in simulator-based settings (Zhao et al. 2021
There is also a substantial body of work on Bayesian LFI, but these methods do not achieve conditional in the sense of Eq. even as B oo (Hermans et al. 2022).
1.1.3 Conformal Prediction
Conformal methods have recently emerged as an approach using data obtain valid prediction regions
under minimal assumptions (Vovk et al. 2005 Shafer & Vovk 2008 Angelopoulos et al. 2023).. Specifically,
given exchangeable labeled data {(X1, Yi)..... .......... (Xm, Ym)). these methods create prediction regions R(x)
such that
P P(Ym+1 E R(Xm+1)) 2 - a.
(3)
The first step a standard conformal method involves creating a conformity score : x y + R, which measures how well the outputs of regression model predict new labels Y E y. In a regression setting, standard choice for the conformity score is the regression residual, given by 8(x, y) ly E|| The prediction region then takes the form R(x) = {v 8(x,y) S (), where / is chosen to be the adjusted (1 (1-a) (1-a)-oug ---annilln of the residuals 8(x,y) y) evaluated on a holdout set not used to train s.
The function т used to construct confidence sets can be interpreted as a conformal score 3, and O can be interpreted as Y. However, statistical theory is typically employed to design 7 in a way that yields optimal confidence sets, leveraging the known distribution f(x 0). Therefore, standard conformal scores are suboptimal for designing 3 for confidence sets. Additionally, most conformal scores only deal with univariate Y, whereas 0 is generally multivariate (sce however Dheur et al. (2024) for approaches that be used for multivariate Y's).
Another key difference between prediction intervals and confidence sets is that 0 is not random, whereas Y is. Thus, marginal coverage (Eq. 3 is insufficient for statistical inference problems; typically, coverage for all fixed 0's, corresponding to conditional coverage (Fq. D also desired. This is because the distribution (0), used sample 0 on the training sct,, often arbitrary; it docs not encode any randomness observed in the real world. Unfortunately, it is well-knoow that conditional coverage can't be achieved without strong assumptions (Lei & Wasserman 2014). While many conformal approaches aim to achieve conditional coverage asymptotically, they only control coverage conditional on the features x, and not y. The exception
to this are label-conditional conformal prediction methods (Vovk et al. 2014 2016 Sadinle et al. 2019 Ding
et al. 2023 but these apply only to classification problems.
Finally, we note that conformal sets have recently been applied to statistical inference (Patel et al. 2024 Baragatti al. 2026 [Cabezas et al.) 2025a). While these methods are highly useful, they are designed for Bayesian framework, where the focus is on controlling marginal coverage over the prior distribution.
5

---

Published in Transactions on Machine Learning Research (04/2026)
1.1.4
Partition- Brriitoon-Based Conformal Methods
Our approach to asymptotically controlling conditional coverage closely related to the works of Vovk et al.
(2 Lei & Wasserman (2014); Boström & Johansson (2020 Tzbicki et al. (2020) 2022 VVok al. (2022): Santos al. (0026). These methods first partition the feature space and then calibrate t using the conformal approach described in the previous section for each partition clement separately. In particular, TRUST is
closely related to LoCart (Cabezas et al. 2025b), which creates data-driven partition of X using regression
trees designed to ensure that
P PYn+1 E R(X++1) Xx+1) ~ a.
However, inst ead of partitioning the feature space X, we partition the parameter space to achieve coverage conditions on 0.
1.1.5 Conformal Predictive Distributions
Conformal Predictive Distributions (CPDs) are a recent extension of the conformal prediction framework
(Vokk al., 2019; 2020 Jonkers et al. 2024 These aim model the entire predictive distribution Y
conformal $ isotonic and balanced (Vovk et al. 2020), the CPD for new test a non-parametric way.
instance X is defined
Q(( = 72 Σ T(S(X, S 8(X+11, + "
(4)
for each E and where u ~ (0, serves as correction term to ensure the continuity of Q((nn+1: y). u).
In simple terms, Q provides an estimated probability distribution for the test labels y based on conformity scores. Specifically, if the data is i...d., Q is valid, meaning that Q((X,+1.9) y), u) follows uniform distribution (Vovk ct al. 2019 and thus Q is valid cumulative distribution function. Leveraging the validity of Q, construct 1 confidence CP prediction intervals by utilizing the specific a/2 and 1 a/2 percentiles of the CPD [Boström et al. 2021).
In this work, we use Conformal Predictive Distributions to estimate the distribution of the test statistic T(X,0) 0). This approach provides the desired distribution-free properties when estimating Cg. Additionally, our CPDs allow for the computation of p-values for hypotheses of interest. However, directly using the definition from Eq.. 4 for statistical inference problems will not lead consistent intervals for any fixed 0. The percentiles of Q would not depend on 8, which is the same issue discussed in Section 1.1.3] To address this, we employ local CPD based on partition of O We note that Boström et al. (2021) also constructs CPDs based on partitions, but these partitions are made in X' rather than in O. While this approach is valuable for prediction tasks, it does not address the problem of statistical inforence.
2 Methodology
Fix a test statistic T, and let H-100) denote the distribution of +(X, 0) given 0, that is, for each To E R, let
H( = : (T 0).
Now, notice that for the confidence set from Equation 2 to have a (1 - a) confidence level, Co must satisfy
a = P(0 E R(X) ()) P(T(X 8) 0)
In other words, we need P(r(X, 8) < Co) a, which implics that Co connocted to the CDF H by
C Ha(())
where H-1(-10) denotes the generalized inverse of H(-10), given by
(A) inf E R H(1)) 2 A).

---

Published in Transactions on Machine Learning Research (04/2026)
This suggests derive confidence sets by estimating H and then using the plugin cutoff
Co.B (a
where HB is the adjusted empirical distribution function defined as
BB(( B ++1
Σ C1((((10) [1((((00))00+1)
The estimated confidence will then be
(x(X):== (000=00000XX0000 20. Co.
The estimated conditional CDF HB can also be used to test statistical hypothesis of the type Ho 0 = 80. Specifically, p-values can be estimated via
A Hx 0o)10)).
where Xobs represents the observed data
We build on this by introducing a novel approach to estimate H-10), which ensures robust properties.
2.1 Estimating H(-10)
Our first estimator of (-10), TRUST (Tree-based Regression for Universal Statistical Testing), is constructed by partitioning O. First, we show how any partition θ can be used to form an estimator of H. Following that, we discuss how optimally select such a partition.
2.1.1 Partition-base Estimate
Let A be any fixed partition of θ and {(1,, X1)........ (0n, } be the simulated dataset from the statistical model. The cumulative distribution function (-1)0) can be estimated using the empirical cumulative distri- bution the 0 values from the simulated set that fall into the same partition element as 0. Formally, for each R, we define
A) = [Ia()] +1
1
Σ
T(T(X,,
00) st))+++1)))))11
(5)
bELACO)
where A(0) represents the element of A containing 0, and Ia(0) = { (1......)) 0, E A(0)) is the set of indices for all 0g values that fall into A(0).
With this partition- based construction of f, the plugin cutoff Čo, (a)(), used to estimate confidence sets, corresponds the adjusted a-quantile of the values {r(X,, 0b) E LA(o). Also, H8 is essentially conditional predictive conformal distribution (Section 1.1.5). with the exception that do not add the uniform distribution used to ensure continuity as it is not needed here. This method therefore aligns with partition-based conformal approach. However, unlike standard partition-baser conformal methods, the partition A must be selected to approximate Haa()) well. The next section discusses how to choose A to achieve this goal.
2.1.2 Choosing the partition: TRUST
For each 0, our goal is to estimate the cutoff C. = Ha 0) so that P(r(X,0) < C. 0} With a finite simulation budget, estimating this quantile separately for every 0 is noisy, so TRUST pools simulations across
Indeed, under the null hypothesis, P (X (o))) S alfo) P (T(Xoba 9b) < H (a)0o) (8o) H #(# (H)) and therefore B ( (x) (0)) will be valid p-value Ff consistently estimated B

---

Published in Transactions on Machine Learning Research (04/2026)
parameter values that induce similar distributions of 7). Concretely, TRUST learns data-adaptive partition A of O and, for a target 0, computes Co,B as the adjusted empirical a-quantile of the r-values whose 0, fall in the same cell A(0) E A. The partition must balance (i) having enough points per cell to stabilize quantile estimation and (ii) keeping cells homogeneous enough that pooling does not bias the cutoff.
To choose A, TRUST fits a regression tree to the simulated pairs (0,, ((x,, (6))== and uses the induced leaves as the partition elements. Intuitively, the tree creates cells A within which the conditional law of T(X,0) 0) varies slowly with 8, so that the pooled empirical CDF in (5) provides an accurate proxy for H f). With such partition, the guaranteed local coverage P(0- E R
(X)
|0- E A(0)) =1 serves as a good
approximation to the desired conditional coverage P(O (x 2006 Theorem 1).
0} = 1 - Meinshausen & Ridgeway
Although our theoretical results assume that the data used to construct the optimal partition is separate from the data used to build the empirical CDF in Equation 5 (refer to Section 3 for details), in practice, we do not perform this data split, as it did not yield any performance gains in our experiments. Algorithm [ outlines the complete TRUST procedure, and Figure provides a graphical visualization of it.
Algorithm 1: TRUST algorithm
Data: Simulated dataset 11,00),... (X,, On); grid Ogrid C θ
Result: The confidence set (X) compute I (01,7(X,,01))
(0n, 7(n,,(nn, n, 0n)):
split I in Itrain and Teal, where Itrain 0 NI 0;
fit decision tree Ò with Itrain: [(0) = B 0) A];
9 will create a partition Auain = (A A of O; for 0 E θ grid do
A(0) < element of Acrain where falls;
JA(Ø) - E [Toa]] 0, E A(0));
Co,B +- adjusted a-quantile of (T(Xes (r(Xe,0c))eed (e)
end
return (X) (xx)=fp f000 E Ogrid T(X,0*) 2 Co-,.
2.2 From trees to forests: TRUST++
A single regression tree can yield partitions that are sensitive to small perturbations of the simulated data. To stabilize the partitioning of θ, TRUST+ uses an ensemble of trees to define data-adaptive notion of similarity between parameter values. However, using standard random-forest machinery to estimate conditional quantiles for example by averaging tree-wise leaf distributions or by computing tree-wise cutolfs Ca and averaging them across trees does not, in general, preserve the distribution-free guarantees of partition-based conformal constructions: once the cutoff is averaged across trees, it is no longer an adjusted order statistic computed from an exchangeable set within a fixed cell, and the usual partition-based validity argument. no longer applies (Cabezas et al. 2025b
((k
Instead, TRUST++ uses the forest only to define data-adaptive notion of locality O. We first fit a random forest with K regression trees, each trained on bootstrap sample of the simulated pairs {(0,, 7 r(x,,6)) (.))) b= Let p(0, f*) denote Breiman's proximity measure (Breiman 2001), defined as the number of trees for which 0 and 0' fall in the same terminal node. Large p(0,0') indicates that many independently grown trees repeatedly group 0 and 6' together, suggesting that 7(X, 0) has similar conditional distribution at these parameter values.
Strict partition (distribution-free version). We first define partition A induced by the equivalence relation 8 ~~ 8' + p(0,0) = K, i.c., 0 and 6' belong to the same cell iff they share a leaf in every tree. We then estimate H(: 0) by the partition-based empirical CDF in 5p and set Ca, मे a (ax 0).
8

---

Published in Transactions on Machine Learning Research (04/2026)
Simulate dataset (X1,0),,-. (X 0p)
Compute
I (8,, 7(X,, 6.1) 10 xX 0n))
Terai
Tral
Fit
Az As
Ör = a-quantile of (r(X,, 10)1 A.Jo.e...
s ()) 0
Partitions induced by
Rx(X)= = f0r =01=((x00)) Cr
Figure 2: Schematic flowchart of TRUST. Over simulated dataset (x,,0),.... (XB. 0g), compute the statistic and create the set I, which is splitted between Itrain and Teal- Itrain is used to train regression tree that predicts T r(X, 0) using 0. The tree partitions the paramctric space in A Ag. If a 0", from the grid, falls inside the A₁ partition then, in order to calculate the cutoff Cgr,B. will compute the adjusted a-quantile of all r(X,0) from which 0 belongs both to Tent and A Finally, we use the calculated cutoff to find the confidence set Rn.
Relaxed neighborhoods (practical version). In practice, often use the neighborhood A(0) {0' E O : p(0, 0') 2 M). where M E {l....,K} K} controls how aggressively pool simulations across 0's. For M < K, A(0) need not induce partition and exact finite-sample coverage is not guaranteed (Guan 2023 but we find it performs well empirically; see Section for how we tune M.
TRUST++ can also be framed as a specific instance of the LCP framework by utilizing a box kernel with the Breiman proximity measure replacing the commonly used Euclidean distance. This adaptation allows leverage the conformalization procedures from Guan (2023) and Hore & Barber (2023) transform TRUST++ into proper conformal method, thereby ensuring exact coverage control. In this setting, the tuning parameter M serves kernel bandwidth, regulating the granularity of the partition A. Larger M yields smaller, stricter neighborhoods (less pooling, lower bias, higher variance), while smaller M yields larger neighborhoods (more pooling, higher bias, lower variance).
Algorithm 2 summarizes the non-conformalized version our method, which we name TRUST++, and Figure shows the flowchart of the procedure.
2.3 Nuisance parameters
Many statistical models contain parameters that affect the data distribution but are not of direct interest. Write the full parameter as = (n, v), where E M the parameter of interest. and и E N is a nuisance parameter. Our goal is to report a confidence set for μ only, while maintaining validity uniformly over the nuisance parameter.
Oracle cutoff for μs. Fix a test statistic r(X, (larger values indicate higher plausibility of p)) For each (H, let
H(t
St p.v), C= p.
If we were interested in inference for the full parameter (H.V), we would use the cutoff Cp. However, guarantee coverage for all nuisance configurations while reporting uncertainty only for le, we must protect
9

---

Published in Transactions on Machine Learning Research (04/2026)
Algorithm 2: TRUST+ algorithm
Data: Simulated dataset (X1, .1)..... (X,, 0B); a grid @grid C Result: The confidence set (X)
compute I = ( (()), (OB, TX BB))
split I in Itaain and Teal, where Itrain n Tcal = 0; draw K bootstrapped samples from Ieaain; fit K TRUST trees, each one with different sample (Alg.
for 0 E O grid do
A(0) + 10' E θ p(0', 0) 2 ,,, where p(0', 0) is the Breiman's proximity measure and M a tuning
parameter;
JA(Ø) - E [Taa]] 0, E A())):
Co. t adjusted a-quantile of (T(X., 0e) DEE)
end
return (x) = -f0- E @grid 7(x,0") 2 Cor
Simulate dataset
(X,,0) (X,,
Compute
I - (X,, ( (Xx,
T
K sumples
Fit
A08) c ( 2
A(8")
X TRUST trees
Ra(X): - c 20)
Go. = a-quantile of (r(X., 0, A(P")
Figure 3: Schematic flowchart of TRUST++. Over a simulated datasct (X1, 61).............. (XB, 0B), we compute the statistic 7 and create the set I. Then, draw K bootstrapped samples over the split Irrain and each sample to fit different TRUST tree. The set A(0) is caleulated using Breiman's proximity measure, P, which defines whether two 6' belong to the same partition element. The cutoff Čø- B then calculatod as the a-quantile of all statistics of their respective that falls over A(O), for the 0's that belong to Teat- Lastly, the estimated confidence set RB can be calculated using Čo.,p B.
against the worst-case v, leading to the oracle cutoff DDalmasso et al. 2024 Izbicki 2025
C₂ inf Cu,.. DEN
(6)
10

---

Published in Transactions on Machine Learning Research (04/2026)
The resulting confidence set is
R(x) = M 2 Ca).
Estimating C₁. with TRUST / TRUST++. We estimate Cu, using the same machinery as before, but now treating (1, v) as the covariate. Given simulations {((Hb, 1b), Xx) we fit a tree/forest regressing T(Xo. (b) on (Js, 1)), which induces a partition of = M N into cells (leaves).
For any leaf A E A, define the leaf-wise empirical CDF (as in 5 and its adjusted a-quantile; denote this leaf cutoff by 9Λ- Then, for given (H,2), estimator satisfies
CuMBB QA(p,))
i.e, it depends only on the leaf A( containing (H, v). Finally, estimate the muisance-robust cutoff by
Cn,B inf Cue.B.
(7)
Why the infimum becomes finite minimum. Because CH.,B is constant within each leaf, the map V + Op.vnn piecewise constant when μ is fixed. Hence, the infimum in reduces to minimumn over the finitely many leaves intersecting the slice {u} N:
C= = min U AEA, An(u)) x N) + }).
This replaces a potentially expensive continuous optimization over v by a discrete minimization over a small set of candidate leaves.
Example 1 (One-dimensional nuisance parameter). Assume is one-dimensional. For a fized μ., the tree induces finitely many intervols in v (e.g., v 81, $1 $1 V < 82, s2), each corresponding to leaf with constant cutoff q, 92 93- Then
inf G min(gr, 42 vat.
so compuling Gy.B only requires evaluating one cutoff per relevand leaf.
Efficient candidates for TRUST++. For TRUST++, the same principle applies: ČZ..B changes only when (n, v) crosses a split threshold in one of the trees. A practical strategy is therefore to build small candidate set nuisance values from the split thresholds. Let A C N be the set of muisance-corrdinate split values appearing in the fitted forest (optionally restricted splits near the root). Construct candidate grid
G = (vte:ve
where is chosen small enough so that v-e and + fall on opposite sides of each split (e.g., one third of the minimum coordinate-we wiste separation among distinct split values). Then, for each fixed μ, we approximate by
Č,.B
min Cp..B. VEG
This avoids gridding N directly (which becomes infeasible in moderate/high dimensions) and leverages the fact that only split-induced regions matter for the leaf-wise cutoffs.
2.4 Uncertainty About the Confidence Sets
Our confidence set (x) (0 E θ -(X,0) > Co} is an estimate of the oracle set R(X) = (0 e θ r(X,0) 2 Ca}, as is the estimate from the other approaches described in Section 1.1 In this section, we explore another advantage of TRUST and TRUST++: They offer computationally efficient way to approximate the uncertainty associated with these sets, which we describe in what follows.
Recall that, for each Co is estimated by computing the adjusted a-quantile of the TA()) {r(Xo, 6b) b } while Co itself is the a-quantile of the distribution of r(X, Let A(le) be the i-th order
(()
11

---

Published in Transactions on Machine Learning Research (04/2026)
statistics of Ta()) and define Z 16 E Ta(e) 8 I Cahl. where Z represents the number of statistics in Ta(S) that are less than or equal to the cutoff Co. Note that if the statistics in Ta(e) are identically distributed, then Z ~ Biinoiia(m, 1- 3). Using this result, we adapt the method by Hahn & Meeker (2011)
for constructing confidence intervals for quantiles to create a - confidence set for Cg. This confidence set is defined as:
A(yy A(())
(8)
where 1 and u are selected to satisfy constraints ensuring both the validity and informativeness of the confidence interval. Specifically, u and are selected such that S S MAcc are as close as possible and satisfy the restriction
(9)
yielding a valid confidence interval. This is because
P(TRO) S Co S x -
Thus, by choosing and as narrowly as possible sal the restriction from Eq. g we get the thinnest valid confidence interval for Cg. Additionally, despite the i.id assumption about statistics in Ta(Ø) being not strictly true, the values in Ta(Ø) are approximately identically distributed due the way the partition was chosen (Section 2.1.2p.
For each θ, let (h) represent the resulting confidence interval for Cs. The uncertainty can propagated to the estimated sets by calculating the following three sets:
I(X) {0 E θ (X, 6
O(X) ( S C.). and
u(X) 10 =
0 S rXx 0) C/
where I(X) represents the set of parameter values confidently inside the confidence interval, O(X) contains values confidently outside, and ((X) includes those where the status remains uncertain. In the terminology
of 3-way hypothesis testing (Berg 2004, Esteves al. 2016 Izbicki et al., 2025). I(X) is the acceptance
region, O(X) is the rejection region, and u(X) is the agnostic region. The size of (XX) be decreased by
increasing the number of simulations used to estimate Cg, B. The next theorem shows that this approach controls the probability of incorrect conclusions.
Theorem 1. Fir x E X and 0 E @. Let ( ( represent a (1 - B) -leel confidence interval for Co such that P (Co < C..) P (ōy < <Co). Then
(0 I(x) & R(x)) < 8/2
and
E R(x)) s 3/2.
This method be readily extended to the setting with nuisance parameters. Let. 17" be the configuration of nuisance parameters for which Cn.B is computed (Equation 7 By construction, Čp.B is the adjusted a-quantile of the set TA(un)) fr(X,, Pb, 76) : IA Thus, we can derive confidence set for the optimal cutoff using the same approach as in Eq. 8 based now on the set TA(ur)
3 Theoretical Results
The key aspect our method lies in the fact that achieving a good approximation of H--)6) allows the partitions in TRUST and TRUST+ to effectively capture the local behavior of the test statistic T.
Building on this, our theoretical framework relies on ensuring that HB closely approximates in both TRUST and TRUST++ This idea formalized in the following assumption:
12

---

Published in Transactions on Machine Learning Research (04/2026)
Assumption 1. Let H represent the opprozimation of H under either TRUST TRUST++, For any and 6 > 0, there erists Bo E N such that, for all B 2 Bo,
P sup EER
MB((10') -(((1)))1((())) SE 2 21-8.
This assumption ensures that, with high probability, He approximates H as increases. This result is supported by thcoretical findings, such as those of Mcinshausen & Ridgeway (2006): Biau al. (2008). the consistency of base models. To guarantee this consistency, certain conditions regarding the distribution of covariates and the structure of the trees necessary. In tree construction, it is important that node sizes are balanced, with the proportion of observations in each node decreasing the total number of observations grows. Additionally, each variable must have a minimum probability of being selected for node splitting, and splits should ensure a balanced distribution of observations between subnodes. These assumptions are quite reasonable and enable the conditional distribution estimates to be consistent with the truc distribution, providing a solid thcoretical foundation for our method.
3.1 Partition Coverage Guarantees
given any fixed partition of O, the cumulative distribution function H-100) can be estimated empirically using the values of 6, that fall within the same partition clement as Although a data-agnostic partitioning approach does not guarantee that H will closely approximate H, we can st.ill ensure that the plugin cutoff Čœ,B = d corresponding the a-quantile of the values {r(X,, 0s) b La(o) J achieves the desired coverage when conditioned on the partition element. This result is formalized in the following theorem.
As discussed in Section 2.1.1
Theorem (Partition-Based Coverage). Let (1),,)) (0p, XB)} be iid. simulated dataset with fized partition A of O where each Og is drawn from reference distribution r(0) and each X, is generated according to the statistical model with parumelers On. For a lest statistic T(X, 0), consider the confidence set constructed by our method:
Co.B), where Co,B- = Ap represents the adjusted a-quantile of the values (T(X,, 0,) :b E TA(e)), conditioned on 9 belonging to the partition element A(O). Then, we have
P(O E Rx(X) 0 A(0)) 21-0,
ensuring the desired coverage probability of the confidence set, conditioned on the partition element.
This result applies broadly to any partition, yet it carries particular significance for those formed using TRUST and TRUST++. For these mothods, we specifically anticipate the relationship
1 a P Rn(X) (AO)) ~ IP (o Rx((x)1))
which suggests that, with these methods, the probability that 0 lies within the estimated region (x(X). given its association with A(0), closely aligns with the nominal confidence level (1-a) a). In the following section, will state the theorem formally. Also, notice that although the local guarantee depends the choice of r(0) (and thus has a Bayesian flavor), the conditional one does not.
Beyond the conditional coverage established above, these results also imply marginal coverage:
Corollary (Marginal Coverage). Let {(0,, Xi)...., (0n, Xn)) be an i.idd. simvllatdd dalasel with a fixed partition A of where each 0, is drawn from reference distribution r(0) and each Xa is generated according to the statistical model with parameters fs. For test statistic 7 7XX, 0), consider the confidence set constructed methods. Then PPO E RX))) 2
13

---

Published in Transactions on Machine Learning Research (04/2026)
3.2 TRUST and TRUST++ Conditional Coverage Guarantees
Both TRUST and TRUST++ employ regression trees that satisfy the consistency Assumption 1 as guaranteed by established results for tree-based models (Meinshausen & Ridgeway 2006 Biau et al. 2008). With these partitions, we not only guarantee partition-based coverage, as indicated in Theorem 2 but also cnsure that both methods asymptotically achieve optimal conditional coverage, as formalized in the next theorem.
Theorem 3 (B-Asymplolic Conditional Coverage). Let {(0), Xi)...., (On, (0n,Xn) Xn)) be an i.idd simalated dataset, where each 0, is drown from a reference distribution r(0) > and each Xs is generated according to the statistical model with parameters 0,- For test statistic 7 (X, 0); consider the confidence set constructed by either TRUST TRUST++ with M K
Rx(X) = (0 Co,B)
where Ca is the culoff delermined by the partition, representing the adjusted a-quantále of the distribution of T(X,, 06) values within the partition containing 0. Then, if Assumption holds, both TRUST and +RUST+ are asymptotically consistent, that is:
lim P P( Rx(X) = -1-a a. 80000
As described in Section 2 in practice we may use relaxed neighborhoods controlled by M. When M < K, these neighborhoods need not induce partition of O, exact finite-ammpl coverage is not guaranteed (uuan 2023 However, in the asymptotic regime B + oc, it is natural to consider the partitioning case M = K, since the partitions are then expected to be sufficiently populated and hence non-degenerate; this helps explain why our asymptotic analysis is predictive of the behavior observed for the relaxed (M < K) version in the experiments.
Theorem 3 shows that the partition constructed by TRUST and TRUST++ designed so that local coverage closely approximates conditional coverage. Moreover, unlike conventional asymptotic approaches, which typically rely on the sample size of the observed dataset x, n, our approach is independent of n. Instead, it depends solely the number of simulations B, which can be scaled up given sufficient computational resources. This framework enables both TRUST and TRUST+ to achieve robust, distribution-free guaranlees and asymptotically attain optimal conditional coverage as B increases.
In the appendix, we discuss the koy results required for the thcorctical framework, provide intuition for why our methods work, and present the proofs of the main results.
4 Implementation Details and Tuning Parameters
To implement both TRUST and TRUST++, we use the efficient decision tree and random forest implementations provided by scikit-learn (Pedregosa al.] 2011).. Since our approaches use regression trees to partition O, managing tree growth is crucial to prevent empty or redundant partition elements. We achieve this through a combination of pre- and post-pruning: in both methods, pre-pruning is executed by fixing the min_samples 1eaf sxperparameterr-typiiie well-populated to produce accurate local estimations of (-10). Specifically, we set 150 and 300 as default samples for TRUST and TRUST++, respectively, throughout all main experiments (Sections 5.1 and 5.2
to 100 or 300 samples to ensure that each leaf is sufficiently
While pre-pruning provides a bascline for leaf stability, TRUST further refines its partition through cost- complexity post-pruning, which balances complexity with predictive performance by removing less informa- tive nodes. In contrast, we omit post-pruming in TRUST++ to preserve the diversity across the ensemble, excessive pruning can diminish the benefits of aggregating varied partitions. For all other hyperparameters, we adhere to scikil-learn defaults, with the exception of the random forest size, which we set to 200 trees to ensure a robust ensemble.
The impact of these structural choices is further explored Appendix A.6 where our sensitivity analysis (Figure 15) reveals that sain_sapple____mmeea leaf is the most consequential hyperparameter for TRUST+ perfor- mance. While this value could scale with sample size, found that fixing it at substantial level, such
14

---

Published in Transactions on Machine Learning Research (04/2026)
300, effectively stabilizes the model. This stability allows the tuning focus to shift toward the neighborhood parameter M, yielding the robust results displayed in Section 5 Notably, TRUST avoids these sensitivity issues because its cost-complexity post-pruning step acts as an automated regularizer, naturally restricting hyperparameter selections toward those that optimize predictive performance.
Shifting the focus this neighborhood selection, our default configuration for the tuning parameter is M K/2, placing TRUST+ in a majority-vote regime. In this setup, 0' is included in the neighborhood A(0) only if majority of trees assign both points to the same leaf. Although this intuitive choice performs well across the LF1 problems discussed in Section 5.2 it is not universally optimal, as some tasks may necessitate larger neighborhoods to compensate for the potential approximate non-invariance of the underlying statistics.
To address this, we propose a straightforward grid-search algorithm for optimizing M using a small, additional simulated validation grid. The main idea is to select M from a fine grid between and K that minimizes estimated deviation from conditional coverage. This is done by calculating coverage across the validation grid with a batch of statisties simulated al each fixed grid point and then computing the mean absolute error of the estimated coverage relative to the nominal confidence level 1-a. Section 5 provides further details on this process. Alternatively, coverage can be estimated using the LF21 diagnostic module (Dalmasso et al, 2024), which also leverages an additional simulation set. Algorithm 3 in Appendix procedure for M.
B outlines the tuning
5 Applications
In this section, we compare the coverage performance of TRUST and TRUST++ (both tuned and majority- votes versions) to the other state-of-the-art competing methods on tractable likelihood, likelihood-free free,, and nuisance parameter settings.
To assess performance, we use several statistic and sizes and simulation budgets B in both likelihood-based and likelihood scenarios. In the nuisance parameter setting, we benchmark our methods with two examples: one using likelihood-free inference and another based on likelihood. We set a common confidence level of 1 - a = 0.95 across all experiments. Comparisons are made against the following bascline mothods:
combinations across multiple sample
Gradient Boosting Quantile Regression (Boosting): implemented in the scikit-learn library (Pedregosa et al.] 2011), we use it to estimate Cg through the (1 a) conditional quantile of the T(X, 0) given 0 DDlmasso et al.) 2020). We fix the maximum depth as 3, the tolerance for early stopping as 15 and the maximum itcration as 100. The remaining hyperparameters are fixed as scikid-leurn'' 's default. To control tree growth in each ileration, we limit the maximum depth to 3 (instead of the default, which is unlimited) and employ a weak learner ensemble approach, which is well-suited for boosting methods (Freund & Schapire 1997 Hastie et al. 2009).
Monte-Carlo (MC): implemented with an equally spaced grid over θ, we simulate nac statistics for cach grid element and estimate C using the (1 a) -qqantiee of these simulations at the closest grid point. For multi-dimensional O, the grid is formed by combining equally spaced one-dimensional grids obtained along each coordinate of θ. To ensure comparability with other methods, the uni-
dimensional grid size is set to 1/d where d = dim e. This guarantees that the Monte-Carlo simulation budget in multi-dimenniooal problems will be close to the fixed budget B. We set in all cases nyc 500.
AMC
Asymptotic: this approach relics on classic asymptotic results for certain test statistics. It esti- mates Co as the (1 - --)-quantile of the statistic's asymptotic invariant distribution. Since estimated statistics in the LFI setting lack invariant asymptotic approximations, this method only applied for comparison in some scenarios with tractable likelihoods.
Normalizing-fione (nflow): we use conditional neural spline flows Durrann et al. 2019) es- timate the distribution of the test statistic conditional on 0. Once trained, the (1 a) qu-quane Co is estimated by drawing large sample Bquantile from this learned distribution and computing
15

---

Published in Transactions on Machine Learning Research (04/2026)
the empirical quantile. Further details regarding the architecture and training procedures for this quantile estimator are provided in Appendix A
To evaluate the conditional coverage performance of each approach in both likelihood-based and likelihood- free scenarios, we compute the Mean Absolute Error (MAE) each method's conditional coverage concerning the confidence level (1 - a). We begin by estimating the conditional coverage of cach method through the
simulation of several statistics inside an evaluation grid. Let @riid denote such grid. We compute the coverage for each 0' E grid at level a for any cutoff estimation method Ö as follows:
cover covera(C,6)) (Č,0) = Taam Σ
(7(0' X (00) 5 C
(10)
where Asim represents the number of simulated statistics and X denotes the i-th observation simulated under 0", We then define the MAE of the method's conditional coverage relative to the nominal confidence level (1 a) as:
MAF(C, a)) = O
(0,er - -a)
(11)
Prrid
The MAE serves as an effective metric for quantifying the extent to which each method Õ deviates from the nominal level of conditional coverage.
In the nuisance scenarios, assess performance by measuring each method's coverage deviation from the oracle coverage, defined as follows:
da C,.CC C) 1grid B'e
Σ
|o, co, Ccvr..(..))-.ovvr.......
(12)
where C is the oracle cutoff specified in Eq. 6 This oracle cutoff controls coverage across all parameters main, and muisance by accounting for worst-casc variations in the nuisance parameter. Consequently, we compare each method's coverage against the oracle rather than the nominal level, as the oracle typically over-covers for the parameters of interest to ensure reliable coverage across all nuisance parameters.
We replicate each experiment 50 times for likelihood-bseed, 30 times for likelihood-free, and 15 times for nuisance examples, and compule the average MAE or deviation from the oracle and its standard error. We identify the top methods in each case as those with the lowest average error, followed by any other meth- ods with performance not significantly different from the best (determined by overlapping 95% asymptotic confidence intervals for the MAE).
In our primary evaluations, the likelihood- brelihood-based scenarios involve one to two parameters, while the LFI settings extend up to five dimensions, reflecting standard inference benchmarks. To assess the scalability of our framework beyond these common regimes, we provide an extensive analysis of how our methods manage increasing parameter dimensionality Appendix A.7 Furthermore, detailed MAF and standard error visualizations for the comprehensive set of benchmark experiments are provided in Appendix A.5
5.1 Examples with tractable likelihood function
In this section, we apply our methods to construct confidence sets on three standard statistical models. We always assume the data is X (Xi,..,,,,)) where the X,'s are i.i.d. To generate the data, use the Normal model with fixed variance, the Gaussian mixture model (GMM), and the Lognormal model with both mean and scale as parameters. For the choice of we explore the following statistics: Likelihood
ratio DDton 2009); Kolmogorov-Smirnot (Marsaglia et al. 2003); Frequentist Bayes Factor (Dalmasso et al.,
2024 Carzon et. al. 2025a) and E-value (Pereira & Stern 1999. The detailed setup is described in Appendix
To evaluate the comparative performance of each method, we consider sample sizes n E {10, 20, 50, 100) and simulation budgets B E (1000, 5000, 10000, 15000) across all statistic and likelihood combinations. While
16

---

Published in Transactions on Machine Learning Research (04/2026)
Figure provides global summary of these results, we specifically highlight the performance for the BFF statistic in Figure Detailed comparisons and heatmaps for the remaining statistics across all experimental settings are provided in Appendix A.5.1
Figures
5 and the supplementary visualizations Appendix
A.5.1 reveal several critical performance
distinctions among the evaluated methods:
n- 10, 1000
20, 1000
n. 50, B 1000
- 100. 1000
10.0
7.5-
H.
5.0
255
0.0
Number simes methed
10000
8 10000
50, 10090
100, 10000
10.0
7.5-
5.0-
2.5
10,
15000
n- 20. B 15000
50. 15000
100. 8 15000
THO T
Siy Someen wi Sa
TRUST
Aue S.SSS
Paum PARIIA
LsouL
Frequency with which each method achieved the best performance (lowest significant MAE) across all statistic likelihood combinations and varying and B. Our methods (in bold) consistently outperform competitors, with a clear advantage in smaller-sample settings (n = 10, 20), where asymptotic approaches tend to underperform. For disaggregated results, see Appendix A.5.1
Figure 4: Tractable Problems:
At B 1000, there are no clear advantage among the methods for different sample sizes n, with TRUST and luned TRUST+ showing competitive results in these scenarios. This balance may be attributed to the limited number of simulation samples, which reduces the ability to differentiate among methods regarding cutoff estimation and overall performance.
Tuned TRUST+ consistently outperforms all competing methods across B from 5000 to 15000 and for all n.
TRUST exhibits good performances for all n, showing a higher count of scenarios with superior outcomes across each combination of and B.
Both Monte-Carlo and Asymptotic methods present comparatively poor performance when the sinulation budget exceeds 1000.
While the boosting method performs well in specific (n, B) configurations, it is generally surpassed by TRUST and the tuned TRUST++. Although these gains are somewhat modest, they are frequent across the majority of tested settiiggs suggesting more robust. behavior from our tree-based approaches. These findings are corroborated by Figures g and 5 as well as the case-by-case analysis in Appendix A.5.I which show that these incremental improvements persist across wide range of test statistics.
Examining the individual statistics (Figure 5 and Appendix A.5.1). our methods show superior performance for the KS statistic, most notably TRUST+ (tuned) and TRUST. For the LR statistic, both the asymptotic approach and our methods achieve satisfactory results. Furthermore, our
17

---

Published in Transactions on Machine Learning Research (04/2026)
n: 10
n:
TRUST++
1:5
0225
2.31
0.033
8.5%
02.0
0.30
2015
5.30
0.501
C.00
tuned
TRUST
03300
10000
10.00
Methods
TRUST
12
12.000
1007
(299))
0,01
10000
12.000
12:00
01001
(2064)
5.000
boosting
LGC
0544
6.50
3628
0006
8017
8.52
0.62
656
3649
327
C003
2015
MC
00081:
11300
12.000
12.00
(100)
0911
51000
18:00)
(1:81
15.30)
5:000
0311
n: 50
100
TRUST++
18
11000:
11:901
(8.00)
(1.00
01100)
(1001)
01100
0.001
12.500
2.000
192001
TRUST
4533
4.09
3007
0.99
0008
8.00
0331
0.00
0.009
8208
8.54
0.01
0010
4.00
2004
MV
Methods
TRUST
0535
2013
C317
0.01
154
3528
boosting
0102
11300
0.000
10000
995
1.00
1000
19002
18
2.00
11022
19911
9.000
MC
0228
4.05
28th
0011
0000
3510
0227
2040
1810
0.67
2004
cone
2012
Intormal: B-1000
Edtormal: $000
dohernal: Bie 10000
Liternali 8w150000
grom: 1000
Be55000
grom Be 101000
grom: B B500
B- 2000
lugnermat Be B-30t0
lemermat 85000
Iaturmati Ba 1000
Ldiermal: Ba5000
Idnormali Ba 10100
dohernat: Bo 15000
gmmm B- 2000
gmm B-3000
prmm B010000
pran: 5000
leproomatt B 1000
lognormatt Bi4000
legrormath Bes 101tog
legnormal: Bae 15000
lognamat:
loynormt:
Benchmarks
Benchmarkss
Figure 5: Conditional coverage MAE heatmap for the BFF statistic across tractable benchmarks. Rows denote the evaluated methods, while columns represent specific combinations of data-generating processes and simulation budgets B. Tuned TRUST++, TRUST++MMV and boosting consistently demonstrate the lowest MAF, representing the most robust choices across the tested scenarios.
methods exhibit good performance for BFF and E-value, with boosting representing a competitive allernative in these settings, mainly in the lognormal scenario.
Based on all comparisons, we conclude that our framework consistently outperforms competing approaches in terms of conditional coverage for several settings.
5.2 Likelihood- Inference Problems
We now apply our methods to construct confidence sets across five well-studied LFI benchmarks, where the likelihood is intractable or no analytical formula. exists for certain statistics, requiring them to be estimated. In cases of intractable likelihood, we simulate the data X = (X1...., X,) using high-fidelity simulator Fg. Given fixed A, we also assume that each X; is i.i.d. The simulators we use are the following: SLCPP (Simple likelihood complex posterior), M/G/1, Weinberg (Hermans et al. 2022); Two Moons (Greenberg ct al. 2019); : SIR. (Lucckmann et al. 2021). We consider the following choices of :: the E- E-Value, BFF, and
Waldo Masserano et al. 2023). A detailed description of them can be found in the Appendix A.112
Since an exact posterior f(0)x) is unavailable in these likelihood-free contexts, we approximate it using a Neural Posterior Estimator (NPE) (Hermans et al., 2022). While we previously introduced Normalizing Flows as baseline for modeling the distribution of test statistics, here utilize distinct flow architecture to directly estimate the parameter posterior. This method approximates f(()x) through an amortized estimator (0)x) built with neural network- based bijective transformations (Rezende & Mohamed, 2015). The estimator is trained on a simulated sample {()) Xi)..... (0p, XB)} generated from each simulator. By distinguishing this posterior estimation from the statistical flow used for Cg calculation, we benchmark the robustness of TRUST++ against varied applications of neural density estimation. Details on the architecture and simulation budgets are given in Appendix A
For each statistic and simulator, we evaluate all methods across combinations of n {1,5, 20, and B E (10, 15, 20, 30) x 103, increasing the simulation budgets relative to Section 5.1 to account for the
18

---

Published in Transactions on Machine Learning Research (04/2026)
greater complexity of these problems. While Figure 0 provides global summary of these performances, specifically include a healmap for the BFF statistic in Figure to illustrate results for this case in the LFI setting. Detailed comparisons of coverage results, disaggregated by the remaining statistics and each data-generating process, are provided in Appendix A.5.2. The results again reveal several performance dist inctions among the methods:
10000
10000
10000
10000
10
15000
20.8 15000
Number of times each methoo performed better
10
20000
10.
1. B
30000
5,8 30000
30000
20,8 8 30000
10
TRUST FURee
TRUSS CIUSTE MU
TRUST
00002 AN
Al A A-2SSAA
Lane
paans PAsII
Figure 6: Tikelihood-free Inference Problems: Frequency with which each method achieved the best result for each statistic simulator combination across varying sample sizes n and simulation budgets B. Our methods (in bold) show consistent superiority when B >> 10,000 and remain competitive even at B = 10,000. For disaggregated results, see Appendix A.5.2
For n 51,5} and simulation budgets below 30, 000, Figure 6 indicates relatively balanced per- formance between tuned TRUST++, TRUST+ MV, and boosting. However, in all scenarios with 20), tuned TRUST++ consistently achieves superior results, while TRUST++ MV outperforms the boosting baseline, albeit by small margin. This suggests that our Breiman distance approach for computing local cutoffs adapts effectively to likelihood-free settings sample sizes and simulation budgets increase.
Figure indicates that tuned TRUST++ outperforms competing methods in nearly all combinations (14 out of 16). Again, tuned TRUST++ exhibits a high count of combinations compared to all other approaches, mirroring its performance observed in tractable experiments.
The Monte Carlo method performs poorly across nearly all scenarios, showing weaker coverage control than other mothods.
The boosting method demonstrates competitive coverage control and consistent performance several scenarios, notably outperforming in the specific case of n = and B 10000. However, in most cases, outperformed by both tuned TRUST+ and TRUST+ MV.
Despite exhibiting poorer coverage control than boosting and tuned TRUST++, TRUST+ MV still shows good performance across all scenarios, which suggests that this default choice for M is con- siderably robust. LFT applications.
19

---

Published in Transactions on Machine Learning Research (04/2026)
TRUST+ tuned
TRUST HV
Methirks
TRUST
unc
boosting
MC
rflo
n: 10
20
TRUST+ tuned
EAT oor ban Aam har Ean
pora han bute han bave Lan Ba 81
TRUST++ MV
55 1417 L40 C4P ba0 bam baa Ban
Macheds
TRUST
boosting
5:0
MC
rflow Bo noo
Core 201e ean
can Lar aee ar
hore hose cau tou
wes Lovo
tos motors Hiloo
aie Reddbod
tan mosoric wiphiigg
two noom wererrng 121pp)
verisbeng 2200g
weticieng E- 1a0eg
Bi Doop
reght fhepp
regh peresso
regt --yoo
tramab Ke 10000
tramaaaa R N00o
ar tiw Dehog
reteovr L200g
two recersi Bi (565g
mover movers 10000
20000
erssdiere X-- I-1 Sohe
eastierg cestborg: Aniziiicg
1,000
eggi oniitargg kest000p
rep mgg 20000
10000
2000
ngt 2-10ppp
trectable 1000p
trectabler 100g
tredaable 2a0so
tar Bie sinsp
velisbeng
Benchmarks
Benchmarks
Figure 7: Conditional coverage MAE heatmap for the BFF statistic across LFI benchmarks. Rows and columns represent methods and simulator bbuget budget combinations, respectively. Tuned TRUST++, T-UST++- MV, and boosting are the strongest competitors. The nflow baseline is specifically competitive for SIR and M/G/1 benchmarks.
In contrast to the results seen in tractable experiments, TRUST demonstrates subpar performance and coverage control compared to both boosting and TRUST+ in likelihood-free scenarios. This indi- cates that the TRUST partition requires enhancements for likelihood-free contexts, a need effectively addressed through the ensembling approach used in TRUST++
The Normalizing Flows baseline demonstrates competitive performance in specific settings, such as the SIR and M/G/1 benchmarks at small sample sizes or when utilizing the BFF statistic, as illustrated in Figure 7 However, across the broader range of experimental configurations, this approach generally exhibits weaker results compared to the tree-based methods.
Appendix A.5.2 and Figure 7 further corroborate the effectiveness of our methods, showing that both TRUST++ tuned and TRUST++ MV perform consistently well across all statistics, with highlights for the BFF statistics. Boosting also remains a good competitor, particularly for Waldo and E- value. Notably, in most scenarios where either TRUST+ tuned or TRUST++ MV attains the best performance, the other method performs within a narrow margin, effectively placing both among the top-performing approaches.
Figure g further illustrates the confidence regions produced by each method alongside the uncertainty quan- tification provided by TRUST++. While TRUST++, boosting, and NFlows all closcly approximate the oracle region, the Monte Carlo method significantly underestimates the set's size. Specifically, the NFlows region appears slightly more conservative than the oracle, whereas TRUST+ and boosting provide tighter approxi- mations. Beyond the primary region estimates, TRUST+ distinguishes itsclf by offering an additional layer of uncertainty quantification. This layer reveals insights into the oracle information that may not be fully captured by the standard region estimators, likely due to constraints in the simulation budget.
20

---

Published in Transactions on Machine Learning Research (04/2026)
True 0
+rust+ tuned
Trust ++ uncertainty
Oracle
-0.2
-0.4
0.2
0.4
Boosting
Monte-Carlo
NFlows
-0.2
-0.4
0.2
0.4
0.2
0.4
0.2
0.4
6
Figure 8: 95% confidence regions and TRUST+ uncertainty for the two-moons example using the e-value statistic (n = 20, = 15,000). Maroon areas denote high confidence, while salmon areas indicate uncertainty, TRUST++ closely matches the oracle set and highlights where additional simulations could reduce uncertainty,
5.3 Examples of nuisance- parametr problems
In this section, we demonstrate the application of both of methods in scenarios involving nuisance parameters. We begin with Poisson counting experiment, originally introduced by Dalmasso et al. (2024) as an example from high-energy physics, where the BFF statistic is non-invariant, requiring both estimation and marginalization. Next, we tackle a classic inference problem: estimating parameter intervals within a gamma generalized linear model (GLM) under the presence of nuisance parameters and limited sample sizes.
We compare our methods to the standard asymptotic approach, which employs a likelihood ratio statistic marginalized over the nuisance parameter. For illustrative simplicity, focus the TRUST++-MV variant in this context. To ensure greater robustness in the more sensitive nuisance parameter regime, set the minimum number of samples per leaf to 550. This increased constraint helps stabilize the local cutoff estimates against the additional variability introduced by the marginalized parameters.
5.3.1 Poisson Counting Experiment
We consider the observed data X = (No, N.) which consists of two counts where N₂ Pois(urb) and Ng Pois(vv + ju). In this example, s, and interest and v is the nuisance parameter. We define 0 = (), E θ = (0,5) x (0, 1.5) and fix the hyper- parameters as 8 = 15, b b= 70 and 7 == 1 to avoid the Gaussian limiting regime for Poisson distributions. We also take the uniform distribution over the parameter space as prior. To derive a statistic T(X, (1) in this case, in this context, we use a marginalized BFF statistic based on the posterior f(u[x), estimated through normalizing flows approach (dctails in Appendix A). We set the simulation budget to B x 103 and 1 to fit all methods.
fixed hyperparameters, while is the parameter of
Although the BFF statistic is marginalized over nuisance parameters, it may still depend on their values. Therefore, we fit all methods considering the full parameter (H, and compute the cutoff for through Eq. 7 Our approaches facilitate this computation by using the tree structure and achieve better results, as shown in Table Table [ighhlghhs the strong performance TRUST++, with coverage closely aligning with the oracle, emphasizing its adaptability to challenges involving nuisance paramcters. Also, we observe that the Monte Carlo method shows substantial deviation from the oracle, further illustrating its limitations in adapting effectively to different inference scenarios.
21

---

Published in Transactions on Machine Learning Research (04/2026)
Table 1: Mean absolute coverage deviation from the oracle for each method in the Poisson counting example. The average across 15 runs reported along with twice its standard error. TRUST+ demonstrates exceptional performance.
Methods
da
SE 2
TRUST
0.0369
0.14 10
TRUST++
0.0041
Boosting
0.0084
0.12-10-1
MC
0.1460
0.11 0.11-10-4
While TRUST+ achieves the best results, boosting also performs well, providing coverage that similarly close to the oracle's. Figure detaiis the differences between the methods. We observe that TRUST+ effec- tively controls coverage deviation across all combinations of (H,V). In contrast, boosting exhibits areas of higher deviation, particularly around 2 for all values of and around 3 when is low. This demon- strates TRUST++'s accuracy in approximating the oracle region throughout the parameter space, indicating effectiveness in estimating the cutoff C₁ defined in Eq.
TRUST+
Boosting
14
14
0.12
Absouuee deviance to oracle
0.10
1.2
12
0.08
1.0
10
0.06
0.8
0.8
0.04
0.02
0.6
0.6
2
2
4
μ
μ
Figure 9: Comparison of absolute deviations from the oracle for each pair of parameters between TRUST+ and boosting. Notably, around 2, boosting exhibits significant deviations from the oracle, whereas TRUST++ effectively controls these deviations, keeping them below 0.1.
5.3.2 Gamma GLM experiment
Consider i.i.d data Y E R+ generated from gamma generalized linear model (GLM) with fixed covariates X, (1,2 (-1,1)3
Y ~ Gamma (1/6, Φ- exp (80 + 31X + 32X,,2))
(13)
where & is the dispersion parameter and 3o, and Bn coefficients for the linear predictor, with e (0, 1.75) and B = (8o, 32) e R², The expected value of each Y is given by:
E|y;] exp (80 + BiXi 82X2).
A standard problem in this setting is to derive a valid confidence interval for single parameter By. A widely used approach to construct an asymptotic confidence interval based on the marginalized likelihood ratio log L(y: 3), where is the MLE, B_, the MLE restricted under a fixed B3 and C(= ; ()) denotes the GLM likelihood defined in Fq. 13 Using this statistic, asymptotic
statistic: LR(y, 33) log L(y: 8 S_
5,) -...
22

---

Published in Transactions on Machine Learning Research (04/2026)
confidence interval can be easily derived by referencing the xi distribution compute C3. in invariant way.
The limitation of this approach is that it relies on large sample sizes to be effective; for small samples, it may fail to produce a valid confidence interval, as illustrated in Figure [b). This is because the marginalized likelihood ratio statistic still depends on the remaining nuisance parameters, B_ and & low-sample regimes. This limitation can be addressed with our approach by computing C3,, using Eq. 7 in a scalable manner. To apply all simulation-based approaches, Truncated Exponential(1, 1.775). For method comparison, sample size to n = 50 and the simulation budget to B = 10-103; with the covariate matrix X fixed according to values generated independently from U(-1, 1) Table shows the mean deviation from the oracle coverage for each method, while Figure illustrates the probability of coverage and confidence intervals for specific values of 83 and Y samples.
assume the priors 8 ~ -N(00 4) (O) and ø
set the parameter of interest to j 1, the
Mean absolute coverage deviation from the oracle for each method in the GLM example. The average across 15 runs is reported along with twice its standard error. TRUST++ demonstrates superior performance compared to all competing methods.
Table 2:
Methods
d,
SE 2
TRUST
0.0312
0.176 10
TRUST++
0.0157
0.163 -163-10-3
Boosting
0.0234
0.171-10-
MC
0.0207
0.157 10-3
Asymptotic
0.0327
By Table 2 we notice that TRUST+ achieves the best performance, with coverage closely matching the oracle and out performing Monte Carlo and boosting by a large margin. Vigure[illustran this proximity for specific values of 3y and observed sample values Y. Although TRUST++ exhibits overcoverage relative to the nominal level 1 - and produces larger confidence intervals compared to other methods, it closely emulates the oracic's behavior, making it more valid than the competing approaches. Additionally, TRUST++'s uncertainty bar indicates that for nearly all parameters within the confidence interval, we can be confident they Iruly lie within the interval, reflecting a well-estimated range. Furthermore, both Table 2 and Figure [ reveal that the asymptotic approach not only deviates more from the oracle but also undercovers and underestimates the confidence interval length. This underscores the limitations of asymptotic methods for small sample sizes and nuisance parameter, even in problems with tractable likelihoods.
6 Final Remarks
This paper introduces novel methods, TRUST and TRUST++, for the distribution calibration of statistical confidence sets, ensuring both finite-sample local coverage and asymptotic conditional coverage when M = K. In practice, M < K also led to good results. By leveraging tools from conformal inference, we create robust confidence sets that extend the applicability of traditional conformal techniques, typically used for generating prediction intervals, to the domain of statistical inference. This adaptation allows us to maintain the desired coverage properties even in challenging settings. In practice, this translates to improved performances across a variety of scenarios, particularly in cases involving small sample sizes n and larger simulation budgets B.
Furthermore, our methods effectively address inference settings with nuisance parameters, notable challenge for other approaches, especially in likelihood-free inference. Unlike existing techniques, TRUST and TRUST+ also enable robust uncertainty quantification about the oracle confidence sets, providing valuable insights into whether additional simulated data is needed to achieve more reliable estimates. Additionally, we can leverage the third branch of I.F2I (Dalmasso et al. 2020 2024h to test for the exact conditional coverage of our confidence sets.
Theorem B demonstrates that both TRUST and TRUST+ are consistent estimators for quantile regression. While we applied these methods specifically to quantile-regress the conformal score 0, their utility extends
23

---

Published in Transactions on Machine Learning Research (04/2026)
far beyond this particular setting. They can be applied in variety of contexts, such as constructing predictive intervals in prediction setting, including for multivariate Y. This capability allows for the generation of label-conditional predictive sets that aim to control IP(Y E (R() Y = y). ensuring coverage conditional on the label.
While our methods are observation-efficcent the sense that they can operate with a small number real observations, they are simulation-based and therefore depend on a simulation budget B. Consequently, if simulator runs very expensive, generating the required simulations may be the dominant computational cost, in many likelihoodffree inference pipelines.
To our knowledge, no other conformal methods aim to control label-conditional coverage for continuous oulcomes Y, making this a novel contribution to the field. This work paves the way for future research to explore label-conditional inference in complex, continuous and multivariate-label settings, potentially improving the reliability of prodiction intervals in domains where conditional control is critical.
Acknowledgments
L.M.C.C is grateful for the fellowships provided by São Paulo Research Foundation (FAPESP), grants 2022/088 7087997 and 8025/06168-8 R. I. is grateful for the financial support FAPESP (grants 2019/11321- and 2023 10706-1) and CNPq (grants 422705/2021-7, 305065/2023-8 and 403458/ 403458/2025-0) /2025-0). R. B. S. producod this work as part of the activities of Fundação de Amparo à Pesquisa do Estado de São Paulo Research, Innovation and Dissemination Center for Neuromathematics (grant 2013/07699-0), The authors are also grateful to Rodrigo F. L. Lassance and Ann B. Lee for their suggestions and insightful discussions.
References
Sara Algeri, Jelle Aalbers, Knut Dundas Morå, and Jan Conrad. Searching for new physics with profile likelihoods: Wilks and beyond. arXiv preprint Xr1i::1011. 10237, 2019.
Anastasios N Angelopoulos, Stephen Bates, et al. Conformal prediction: A gentle introduction. Foundations and TrendsS in Machine Learning, 16(4):494-591, 2023.
Meili Baragatti, Céline Casenave, Bertrand Cloez, David Métivier, and Isabelle Sanchez. Approximate Bayesian computation with deep learning and conformal prediction. Journal of Computational and Graph- ical Slatislics, (1)443 460, 2026.
Nathan Berg. No-decision classification: an alternative to testing for statistical significance. The Journal of Socio- Fecii-Eeonooics, (3:63:331 650, 2004.
Gérard Bian, Luc Devroye, and Gábor Lugosi. Consistency of random forests and other averaging classifiers. Journal of Machine Learning Research, 9:2015 2033, June 2008. ISSN 1532-4435.
Michael GB Blum and Olivier François. Non-linear regression models for approximate Bayesian computation. Statisties and Compuling, 20:63 73, 2010.
Henrik Boström and Ulf Johansson. Mondrian conformal regressors. In Conformal and Probabilistic Predic- tion and Applications, pp. 114-133. PMLR, 2020.
Henrik Boström, Ulf Johansson, and Tuwe Löfström. Mondrian conformal predictive distributions. Conformal and Probabilistic Prediction and Applications, pp. 24-38. PMLR, 2021.
Johann Brehmer, Gilles Louppe, Juan Pavez, and Kyle Cranmer. Mining gold from implicit models to improve likelihood-fres inference. Proceedings of the National Academy of Sciences, 117(10):52422522) 52449 2020.
Leo Breiman. Random forests. Machine learning, 45:5-30 32, 2001.
Luben Cabezas, Vagner S Santos, Thiago R Ramos, Podro LC Rodrigues, and Rafacl Izbicki. Cp4sbi: Local conformal calibration of credible sets in Fomulationbbase inference. arXio preprint arXiv:2508.17077, 2025a.
24

---

Published in Transactions on Machine Learning Research (04/2026)
Luben MC Cabezas, Mateus P Otto, Rafacl Izbicki, and Rafacl B Stern. Regression trees for fast and adaptive prediction intervals. Information Sciences, 686:121369, 2025b.
James Carzon, Aishik Ghosh, Rafael Izbicki, Ann Lee, Luca Masserano, and Daniel Whiteson. On focusing statistical power for searches and measurements in particle physics. arXiv preprint orXiv:2507.17832 2025a.
James Carzon, Luca Masserano, Joshua D Ingram, Alex Shen, Antonio Carlos Herling Ribeiro Junior, Tommaso Dorigo, Michele Doro, Joshua S Speagle, Rafael Izbicki, and Ann B Lee. Trustworthy scientific inference with generative models. ar preprint arXim:2508. 026002 2025b.
George Casella and Roger Berger. Statistical inference. CRC Press, 2024.
Jiahua Chen and Pengfei Li. Hypothesis test for normal mixture models: The em approach. The Annals of Statistics, pp. 22222, 2009.
Kyle Cranmer, Johann Brehmer, and Gilles Louppe. The frontier of simulation-based inference. Proceedings of the National Academy of Sciences, 117(48):30055 -00622, 2020.
Niccolo Dalmasso, Rafael Izbicki, and Ann Lee. Confidence sets and hypothesis testing in a likelthood-Tree inference setting. In International Conference on Machine Learning, pp. 2323 2334. PMLR, 2020.
Niccolò Dalmasso, Luca Masserano, David Zhao, Rafael Izbicki, and Ann B Lee. Likelihood-free frequen- tist inference: Bridging classical statistics and machine learning for reliable simulator-based inference. Electronic Journal of Statistics, 18(2):5045-500 2024.
Carlos de B Pereira, Julio Michael Stern, and Sergio Wechsler. Can a significance test be gemuinely Bayesian? Bayesian Analysis, 311):79-100, 2008.
Morris H DeGroot and Mark J Schervish. Probability and statistics. Pearson Education, 19:33, 2012.
Victor Dheur, Tanguy Bosser, Rafacl Izbicki, and Souhaib Ben Taieb. Distribution-free conformal joint prediction regions for neural marked temporal point processes. Machine Learning, 113(9):7055 7102, 2024.
Tiffany Ding, Anastasios Angelopoulos, Stephen Bates, Michael Jordan, and Ryan J Tibshirani. Class- conditional conformal prediction with many classes. Advances in Neural Information Processing Systems, 36:64555 64576, 2023.
Marcio Diniz, Carlos AB Pereira, Adriano Polpo, Julio M Stern, and Sergio Wechsler. Relationship between Bayesian and frequentist significance indices. International Journal for Uncertainty Quantification, 2(2). 2012.
Mathias Drton. Likelihood ratio tests and singularities.
The Annals of Statistics, pp. 979-1012, 2009.
Conor Durkan, Artur Bekasov, Tain Murray, and George Papamakarios. Neural spline flows Aduances in Newral Information Processing Systems, 32, 2019.
Luis G Esteves, Rafael Izbicki, Julio Stern, and Rafael Stern. The logical consistency simultaneous agnostic hypothesis tests. Entropy, 18(7):256, 2016.
Yoav Freund and Robert F, Schapire. A decision-theoreticc generalization of on-line learning and an application to boosting. Journal of Computer and System Sciences, 55(1):119 139, 1997.
David Greenberg, Marcel Nonnenmacher, and Jakob Macke. Automatic posterior transformation for likelihood-fre inference. In International Conference on Machine Learning, pp. 2404-2414. PMLR, 2019.
Leying Guan. Localized conformal prediction: A generalized inference framework for conformal prediction. Biomelrika, 110(1):33 50, 2023.
25

---

Published in Transactions on Machine Learning Research (04/2026)
Gerald Hahn and William Q Meeker. Statistical intervals: guide for practitioners, volume 92. John Wiley & Sons, 2011.
Trevor Hastie, Robert Tibshirani, and Jerome H Friedman. The elements of statistical learning: data mining, inference, and prediction, volume 2. Springer, 2009.
Joeri Hermans, Arnaud Delaunoy, François Rozet, Antoine Wehenkel, Volodimir Begy, and Gilles Louppe. A crisis in simulattonn based inference? beware, your posterior approximations can be unfaithful. Transactions on Machine Learning Research, 2022.
Rohan Hore and Rina Foygel Barber. Conformal prediction with local weights: randomization enables local guarantees. Xrviv preprint Xrxi:22310. 1.000000 2023.
R. Tzbicki, A.B. Lee, and C.M. Schafer. High-dimensional density ratio estimation with extensions to ap- proximate likelihood computation. Journal of Machine Learning Research (AISTATS Track), pp. 420-429, 2014.
Rafacl Izbicki. Machine Learning Beyond Point Predictions: Uncertainty Quantification. 1st edition, 2025.
ISBN 978-65-01-20272-3.
Rafael Izbicki, Ann B Lee, and Taylor Pospisil. ABC CDE: Toward approximate Bayesian computation with complex high-dimensional data and limited simulations. Journal of Computational and Graphical Statistics, 28(3):481-45) 492, 2019.
Rafael Izbicki, Gilson Shimizu, and Rafael Stern. Flexible distribution-free conditional predictive bands using density estimators. In International Conference Artificial Intelligence and Statistics, pp. PMLR, 2020.
Rafael Izbicki, Gilson Shimizu, and Rafael B Stern. Cd-split and hpd-split: Efficient conformal regions in high dimensions. Journal of Machine Learning Research, 23(87):1 32, 2022.
Rafael Izbicki, Luben M. C. Cabezas, Fernando A. B. Colugnatti, Rodrigo F. L. Lassance, Altay A. L. de Souza, and Rafacl B. Stern. React to nhst: Sensible conclusions for meaningful hypotheses. The Quantilative Melhods for Psychology, 21(2):43 66, 2025.
Jef Jonkers, Glenn Van Wallendacl, Lue Duchateau, and Sofie Van Hoecke. Conformal predictive systems under covariate shift. arXiv preprint 4rX1::2444115018 2024.
Robert E Kass and Adrian E Raftery. Bayes factors. Journal of the American Statistical Association, 90 (430):773-7955 1995.
Erich Leo Lehmann and Joseph Romano. Testing statistical hypotheses. Springer, 2005.
Jing Lei and Larry Wasserman. Distribution-fres prediction bands for non-parametric regression. Journal of the Royal Statistical Society Series Statistical Methodology, 76(1):71-96, 2014.
Jan-Matthis Lueckmann, Giacomo Bassetto, Theofanis Karaletsos, and Jakob H Macke. Likelihood-free inference with enulator networks. In Symposium on Advances in Approzimate Bayesian Inference, pp. 32-53, 2019.
Jan-Matthis Lueckmann, Jan Boelts, David Greenberg, Pedro Goncalves, and Jakob Macke. Benchmarking simulation-based inference. In International Conference on Artificial Intelligence and Statistics, pp. 343- 351. PMLR, 2021.
George Marsaglia, Wai Wan Tsang, and Jingbo Wang. Evaluating kolmogorov's distribution. Journal of Statistical Software, 8:1-4, 2003.
Luca Masserano, Tommaso Dorigo, Rafael Izbicki, Mikael Kuusela, and Ann Lee. Simulator- based inference with waldo: Confidence regions by leveraging prodiction algorithms and posterior estimators for inverse problems. In International Conference on Artificial Intelligence and Statistics, pp. 2960-2974. PMLR, 2023.
26

---

Published in Transactions on Machine Learning Research (04/2026)
Nicolai Meinshausen and Greg Ridgeway. Quantile regression forests. Journal of Machine Learning Research, 7(6), 2006.
George Papamakarios, David Sterratt, and lain Murray. Sequential neural likelihood: Fast likelihood-ffree inference with autoregressive flows. In The 22nd International Conference on Artificial Intelligence and Statistics, pp. 197-848,47 2019.
Yash Patel, Declan McNamara, Jackson Loper, Jeffrey Regier, and Ambuj Tewari. Variational inference with coverage guarantees in simulation- sanslttion-based inference. In Forty-first International Conference on Machine Learning, 2024.
Fabian Pedregosa, Gaël Varoquaux, Alexandre Gramfort, Vincent Michel, Bertrand Thirion, Olivier Grisel, Mathieu Blondel, Peter Prettenhofer, Ron Weiss, Vincent Dubourg, et al. Scikit-learn: Machine learning in python. Journal of Machine Learning Research, 12:2825 2830, 2011.
Carlos Alberto de Bragança Pereira and Julio Michael Stern. Evidence and credibility: full Bayesian signif- icance test for precise hypotheses. Entropy, 1(4):99 110, 1999.
Danilo Rezende and Shakir Mohamed. Variational inference with normalizing flows. In International Con- ference on Machine Learning, pp. 1530- PMI.R, 2015.
Mauricio Sadinle, Jing Lei, and Larry Wasserman. Least ambiguous set-valued classifiers with bounded error levels. Journal of the American Stalistical Associalion, 11 114(525):223 234, 2019.
Vagner Santos, Victor Coscrato, Luben Cabezas, Rafael Izbicki, and Thiago Ramos. Loboost: Fast model- native local conformal prediction for gradient-boosted trees. arXic preprinl or Xiv: 22432, 2026.
Mark J Schervish. Theory of statistics. Springer Science & Business Media, 2012.
Glenn Shafer and Vladimir Vovk. A tutorial on conformal prediction. Journal of Muchine Learning Research, 9(3), 2008
Vincent Stimper, David Liu, Andrew Campbell, Vincent Berenz, Lukas Ryll, Bernhard Schölkopf, and José Miguel Hernández-Lobato. normflows: A pytorch package for normalizing flows. Journal of Open Source Software, 8(86):5361, 2023.
Aad W Van der Vaart. Asymptotic statistics, volume 3. Cambridge University Press, 2000.
Vladimir Vork, Alexander Gammerman, and Glenn Shafer. Algorithmic learning in a random world, vol- ume 29. Springer, 2005.
Vladimir Vork, Tvan Petej, and Valentina Fedorova. From conformal probabilistic prediction. In IFIP International Conference on Artificial Intelligence Applications and Innovations, pp. 221-230. Springer, 2014.
Vladimir Vovk, Valentina Fedorova, Ilia Nouretdinov, and Alexander Gammerman. Criteria of efficiency for conformal prediction. In Symposium on Conformal and Probabilistic Prediction With Applications, pp. 23-39. Springer, 2016.
Vladimir Vork, Jieli Shen, Valery Manokhin, and Min-ge Xie. Nonparametric predictive distributions basod on conformal prediction. Machine Learning, 108(3):445-41) 444, 2019.
Vladimir Vovk, Ivan Petej, Πla Nouretdinov, Valery Manokhin, and Alexander Gammerman. Computation- ally efficient versions of conformal predictive distributions. Neurocomputing, 397:292 308, 2020.
Vladimir Voek, Alexander Gammerman, and Glenn Shafer. Algorithmic Learning in Random World. Springer Nature, 2022.
Supawadee Wichitchan, Weixin Yao, and Guangren Yang. Hypothesis lesting for finite mixture models. Computational Statistics & Data Analysis, 132:180- 189, 2019.
David Zhao, Niccolò Dalmasso, Rafael Izbicki, and Ann B Lec. Diagnostics for conditional density models and Bayesian inference algorithms. In Uncertainty in Artificial Intelligence, pp. 1830- 1840. PMLR, 2021.
27

---

Published in Transactions on Machine Learning Research (04/2026)
A Experiment details
In this section, we provide supplementary details regarding our experimental methodology and present additional results. We specify the statistics and models models/simulatons used for both the tractable and intractable likelihood comparisons and describe the architectures for the LFI posterior estimator and the estimator used in the Poisson Counting experiment.
For implementation, all experiments involving normalizing flows for posterior estimation were conducted us- ing the normflows Python package (Stimper et al. 2023). We employed autoregressive rational quadratic spline architecture for cach flow (Durkan et al. 2019 and the models were trained on an Acer Nitro 5 AN515-54 using a GPU (CUDA).
A.1 Statistics and models details
A.1.1 Tractable likelihood
The data-gencrating processes arc:
Normal model with fixed variance: X, itd N(0, 1), with 0 E e= [-5,5). When a prior is needed to build 7, ~ 0.250.25).
Gaussian mixture model (GMM): X; dd 0.5.V (0, 1) 0.5N with 0 E e [0, 5]. When prior is needed to build 5, use 0 ~ (0..25, 1) the prior.
Lognormal model with both mean and scale parameters: X₁ sdd μ d² E [-2.5, 2.5] [0.15, 1.25] O. When prior is needed to build т, (μ, o²) NIG(0, 2, 1).
lognormal(p, o²), with
To choose T, consider:
Likelihood ratio: Considering that L(:;0)) represents the likelihood function, the likelihood ratio statistic is given by:
C(x: 0o)
LR(x, 0o) logg
(14)
supay40 L(x; 0)
Under regularity conditions, Wilks' theorem Drton 2009) implies that -2LR(XX 0o) = 0g has
an asymptotic X distribution, which is typically used to approximate Cg. However, regularity conditions are not always met, as in the case of the Gaussian Mixture Model (Chen Li 2009).
Kolmogorow-Smirnov statistic: Let (-) be the theoretical CDF under Og and F.(-)) be the empirical CDF estimated using data X with size n. The Kolmogorov-Sminao statistic is defined as:
KS(x, Bg) sup Fe, (x) F.())) D,
=ex
In this case, a classical result used to approximate Co is that, under the mull hypothesis:
VND, "= K.
with IC being the Kolmogorov distribution Marsaglia et al. 2003
Considering prior probability л over O and the comparison of the hypothesis Ho 0 E Oo to its complement H₁ : E O1, the Bayes factor (BF) is given by the ratio of the marginal likelihood of both hypothesis [Kass & Raftery 1995
Bayes Factor:
Ho) Se., C(x; (0)
BF 0o)
(15)
Hi) Je, f(x; ()
28

---

Published in Transactions on Machine Learning Research (04/2026)
where To and represent the restrictions of to O, and 11 respectively. Since focus is on the precise hypothesis Ho 0 = Oo without altering the joint distribution of the data X and 0, we define the restrictions as follows:
1 Pu T
x(0) #
No (
otherwise
otherwise
and then we compute the Bayes Factor statistic as:
Sco f(x: ())
L(x; 00) f(x) (0o) f(x) =(0))
f(0v, x)
f(0o)x)
BF(x, Bo)
(16)
Se, [(x: ()
Following Dalmasso et al. (2024), we use the Bayes Factor as frequentist statistic to construct confidence sets. We adopt the term "Bayes Frequentist. Statistic", introduced by Dalmasso et al. (2024).
E-value: Another statistic used to test. hypotheses in a Bayesian context is the Full Bayesian Significance Testing (Pereira & Stern] 1999: de B Pereira al. 2008). This procedure assigns an evidence measure based on the posterior distribution evidence measure is small. Specifically, let Te, {0 E θ f(0[X) f(0o X)). The Bayesian evidence mcasure (e-value) in favor of Ho is defined as:
the hypothesis Ho and rejects if the
ev(x, 0o) = - 1-PP 11 J. f(0(x)d0.
(17)
Although the e-value is originally considered Bayesian measure, we use it as a frequentist statistic to construct confidence sets. Therefore, we use the term "Frequentist e-value" to refer to this statistic in our context. Diniz et al. (2012) establishes an asymptotic connection between the e-value and the likelihood ratio statistic, providing an asymptotic approximation for the e-value under contour restrictions. The resulting asymptotic cutoff corresponds the significance level for the simulation settings used here.
A.1.2 Intractable likelihood
We consider the following simulators:
SLCP (Simple likelihood complex posterior): in this model, we consider that x xx, E R2 represents 2d-coordinates of points. The coordinates of each point are sampled independently from multivariate Gaussian whose mean and covariance matrix
parametrized by 5-dimensional parameter 0 (Hermans et al.) 2022 Despite the likelihood's 's simplicity, the posterior of this model is complex (Papamakarios et al. 2019), making it difficult to use the BFF and E-value
statistics in the analytical formula. We let θ = (33,3)5 and use 0g ((33,3) as prior for the model and to estimate T.
Two Moons:
here we consider likelihood model with (x1, E IR2 and two-dimensional 0 such that the posterior exhibits bimodal moon shape-like structure (Greenberg et al. 2019). In this case, we must model and approximate the posterior to estimate the BFF and E-value statistics. We set θ = (-1,1)² and use 0; ~ U( (1,1) as a prior for the model and to estimate т.
M/G/1: this model describes processing and arrival queue system, where a 3-dimensional param- eter 0 influences both the service time per customer and the intervals between arrivals (Hermans et. al. 2022). Here, the sample x = (xi,..., E IR consists of 5 equally spaced quantiles of inter-departure times. In this case, the likelihood [(x; 6) is intractable, and computing the BFF and E-value statistics through likelihood-based inference would require high-dimensional intogration [Blum& & François 2010). We set (0, 10)2 x (0, 1/3) and use (01, (01,82, 0z)) UO, 10)2 (0,11/3)
prior for the model and to estimate
29

---

Published in Transactions on Machine Learning Research (04/2026)
Weinberg: this simulator consists of high-energy particle collision physics model (Hermans al, 2022j. Here, our sample x E R is a measure of the Weinberg angle and we are interested in inferring the Fermi's constant 0 E IR. As the likelihood is intractable, we must estimate all kinds of statistics for this problem. We set e = (0.5, 1.5), and use the prior distribution ~ UO055, 1.5) for estimating
T.
SIR: In this example, we examine an epidemiological model that tracks the dynamics of individuals across three states: susceptible (S), infectious (I), and recovered deceased (R) (Lueckmann al., 2021). The sample x (z1, 22, 22) RR represents the count of individuals in each state within a population of 1000, observed after 10 itcrations of the model. The parameter 0, 2-dimensional vector, denotes the contact rate and the mean recovery rate of the model. This case also involves an intractable likelihood, requiring estimation of all relevant statistics. We set = (0, 0.5)² and apply the prior 0, ~ (O, 0.5) for estimating T.
For choices T, we consider both the E-value and BFF introduced in Appendix A.1.1 along with the Waldo statistic (Masserano et al. 2023
Waldo(x, 06) (E(0)x] 0b) V (E 0o)
(18)
where E[0]x] and V[0[x] replace the MLE estimator 0 and its variance by the conditional mean and covariance matrix of given x. As detailed by Masserano al. (2023), under Bayes estimator assumptions, the Waldo statistic retains the saine asymptotic properties the Wald statistic. However, for smaller n, Waldo can
leverage consistent priors on 0 to produce lighter confidence sets. This makes Waldo compelling choice in scenarios where the likelihood is intractable.
A.2 LFI posterior estimator
In the LFT setting, we employed a total of six flows, each consisting of two hidden layers with 128 units per layer. A dropout probability 0.35 was applied to each layer to prevent overfitting, For optimization, we trained the model for maximum of 1000 epochs, with early stopping triggered after 30 epochs of no improvement. The Adam optimizer was used with learning rate of 3 10 and weight decay of 10-5, To train the neural network, we simulated B = 30, 000 samples for all sample sizes
A.3 Normalizing flow cutoff estimation
To estimate the conditional distribution of the test statistics (-10) through conditional normalizing flow, we consider the architecture consisting of a sequence of 4 flow layers, where each transformation is conditioned on the parameter vector 0. The conditioning logic is processed through neural network featuring' 2 hidden layers with 64 units each.
During training, we utilize a simulation budget of B = 10000 and minimize the negative log-likelihood using the Adam optimizer with batch size of 250. To prevent overfitting, we implement an early stopping criterion with patience of 50 epochs. Once the model is fitted, the (1-a) (1)a)-quatill Co is computed by generating B Bquantite 2000 samples from the learned conditional density ((H(0) for each evaluation point in the parameter space and ext exttactig the corresponding empirical quantile.
A.4 Poisson Counting Experiment posterior estimator
For the nuisance parameter experiment, we used a total of four flows with the same configuration for hidden layers, hidden units, dropout probability, and optimizer as in the LFI neural posterior model. Tn this case, training was conducted for up to 2000 epochs, with early stopping triggered after 200 epochs without improvement. The neural network was trained using B 25, 000 simulated samples for n
A.5 Experiment results
This subsection presents additional heatmaps comparing the conditional coverage performance of each method for each statistic separately. In each heatmap, rows correspond to the hods and columns rep-
30

---

Published in Transactions on Machine Learning Research (04/2026)
resent specific scenarios, defined by model and budget combination. The heatmaps are divided by each sample size. For each scenario, we highlight the top-performing methods in green, which are those with the lowest MAE or an MAE within two standard errors of the minimum.
A.5.1 Tractable likelihood detailed results
Figures 5 and 12 show the performance of the BFF, LR, KS, and E-value statistios, respectively, for sample sizes of n = 10, 20, 50, 100.
Overall, TRUST+ tuned and TRUST are strong competitors for tractable models. Nonetheless, certain methods are preferable in specific scenarios. For instance, the asymptotic approach performs particularly well for the LR statistic under the normal model with fixed variance (Figure [O]. Similarly, boosting shows competitive performance for the E-value: while not consistently superior across all setups, its performance improves with larger sample sizes. For the E-value, our methods perform well for n 10, 20, but their advantage diminishes as the sample size increases.
n: 10
20
TRUST+
tuned
TRUST++
812
0.00
3.000
0.044
6.008
3,300
6.20
6.007
3,300
0.000
0.006
8.009
0.366
8.687
8.586
0,015
8.533
8.55
0.36
3.97
3
6:08
3.833
MV
Methods
TRUST-
030
11.001
6.000)
3.855
16.04:1
13 9350
18.04 01.001
00.001
8.605
01
00.001
01004
15 0185
(1004)
13.020
boasting-
10 18830
12.000
0.080
11.011
3.000
0.60
10.001
10.001
01665)
10 888
(6065)
14.00
MC
215
0.009
0.033
0.03
0.03
2111
6:29
0:533
0.00
8.05
0.955
0.013
3.55
3.55
0:009
1110
asymptotic
835
0.006
0.006
3.35
0.08
6.033
3311
6:23
6328
3.28
6335
6335
8.55
0.006
0.055
8.556
0.055
8.59
8.35
0/06
3330
3316
6:38
3110
50
100
TRUST+
tuned
(2.83
100001
60000
13.000
100034
8:000
13:300
10:000
00.000
000001
0.0001
TRUST+
223
3.99
8:008
0.29
0.91%
0.00
0.50
D/06
0.09
2.29
MV
Mechods
TRUST
2.006
0.006
3.35
0.031
8.00
8.00
0,000
0.714
0.008
0333
0.00
boosting
832
(2.001
R.000
0.800
9000
000
50.001
000001
0.004
13 1380
(0000)
895
MC
asymptotic C
2:00
2529
Duze
dreemmal: 1000
Idnormal: 8-5000
Idnormat: 8-30000
Idhormat: B=15000
gmim: B- d0p
gmmm: 5100
gmnee Be10oo0)
smmre Be15000
Jognormal: Be 105g
Ingnermal: 5:000
loyromat B 100000
Ingrormat: 150000
Jdharmal B-1000
drormaa:: 5
Idnamatt B=100000
Idnarmat: B=15000
gmmn: Bw5000
gmmm: Be tw100a
gmnne B-150000
lognarmat: 1000
lognarmal: B-5000
legnomaa B 10000
lognormal: Bir 15000
gmn:: Ba
Benchmarks
Benchmarks
Figure 10: Best methods (lowest MAE) for LR on tractable models. The asymptotic is always among the best competitors for the Normal model with fixed variance D.G.P., and also appears in some setups for the GMM. Again, our methods are consistently good performers, especially TRUST++ tuned and TRUST
A.5.2 Intractable likelihood detailed results
Figures 7 13 and 14 present the models' performance for the BFF, E-value, and Waldo statistics, respec- tively, in the LFI experiments with sample sizes 1,5, 10, 20.
TRUST++ tuned, TRUST++ MV, and boosting emerge as the overall best performers. TRUST achieves reasonable results in certain scenarios (e.g,, Waldo on the SIR benchmark with n = 10) but rarely ranks among the top methods. MC, in contrast, consistently performs poorly for intractable likelihoods.
For n 1, TRUST++ tunod, TRUST++ MV, and boosting achieve strong results across all statistics. Inter- estingly, TRUST performs well for the E-value in the tractable generator, showing good coverage control for B- 15,000, 30,000,30,000. 30,000.
31

---

Published in Transactions on Machine Learning Research (04/2026)
n: 10
n: 20
TRUST++
200e
CO0E
8.55
E/CE
E/CE
3.334
8:005
8:00T
2,325
0.006
0.035
0:005
8.05
8.00
0/06
8.50
2.11
0:00
tuned
TRUST++
MV
R.000)
03830
8:008:
12,000
16,0001
0.000
3.300
16.0091
5.004
0.60
10.000
0.001
10.001
0L0041
13 03.023
(3 0.60)
(6,000)
13 038950
Methods
TRUST
03:32
1C0001
50000)
10.031
5.002
8.0021
0.00
00.000
000001
000001
(0005)
0.90
boosting
2012
0.00
0.03
1.99
2013
022
0.315
0.00
3.09
3.15
3.10
1824
MC
0.013
3.99
0.01
3.99
8.011
4810
5.0000
0.000
0.05
0.015
8007
8.00
0/08
3.11
0.081
1010
asymptotic-
18880
01.081
01000)
12:00
10.0811
0:000
14:000
10.000
0.00
0000
00.041
01001
01801
18500
10
88881
10.00
n: 50
n: 100
TRUST++
2004
2:30
0:166
0:501
0.504
3.505
2.505
2007
2021
tuned
TRUST+
MV
8010
8880
0.027
0:00
01100
0.107
13.90
0.09
8.804
8.809
8.08
030
2004
Mothods
TRUST-
11/0031
01000)
11/0031
01000
01004
19 8800
baosting-1
8.82
8.77
6.26
0.00
6.22
0.67
6008
MC
6008
0.033
3.000
6.066
6.000
3.30
6.00
6.00
0.10
0.50
0.50
0.066
8.50
0.53
0.001
0006
8 8.53
3.29
6008
3011
asymptotic
8000
R.000
R.000)
0.000
100001
13.000
10.000
80001
8000
10.001
2.000
8.000
50000
00000
880
000041
13 880
1 880
(6.00)
13 8300
1dromat 801000
Idturmat Basono
Idnormali B- Torn
Ednomatl: B 55000
grim 8=1000
prem: 8 $t00000
gmmmi 100000
ummn Bes 15000
topnomatl 1000
legnomat Be5000
legnomal: B 100100
legromaal B 15000
Idnomal B=2000
Idrormal 8
Idnamas:: B Loavo
Idrormal: B 55000
prom: Ba 100000
grnm 8 Be50000
grmmi B 100000
grami B 1-10000
legnamat: Bwi0oo
legnurmat Be
legromai B 100s0
logrommel: B- 150000
Benchmarks
Benchmarks
Figure 11: Best methods (lowest MAE) for KS on tractable models. TRUST+ tuned is among the best meth- ods in 47 out of 48 setups, proving to be a powerful competitor. TRUST is also very consistent, appearing in the majority os scenarios. MC and TRUST++ MV have appeared less than the other two, but are considerably more present than boosting and asymptotic, which are not a good choice for this statistic, as shown in the graph.
A.6 Hyperparameter sensibility
To provide comprehensive view of the robustness of TRUST++ we include a sensitivity analysis in this section. For this study, utilize the two moons simulator with simulation budget of B = 15000. Across 30 independent experiment repetitions, we evaluate the influence of three core decision tree hyperparameters on the coverage MAE, while holding the others at their default values: K = 200 trees, min_ sai__saples____ leaf = 300, and max_depte = None.
The configurations compared in our analysis are summarized below:
Number of Trees (K): We compare K E {50, 100, 200, 400, 500, 700), using K 200 as the base default.
Minimum Samples per Leaf: We evaluate values E (50, 100, 200, 300, 500, 750, 1000, 1250}, with a default of 300
Maximum Tree Depth: We test varying depths E (5, 10, 20, None), where None is the default and denotes unlimited growth, being ssigned numerically to 0
Neighborhood Parameter (M): Throughout these tests, M is fixed at K/2, placing the model in majority-vote regime.
Figure 15 displays both the coverage MAE and the computational runtime for TRUST++ across each hyperparameter combination. The results underscore that the most critical performance factor is the min min_samples__e hyperparameter. We observe that very small leaf sizes lead to a significant spike in MAE; however, once the threshold reaches approximately 300 samples, the performance plateaus. This
32

---

Published in Transactions on Machine Learning Research (04/2026)
n: 10
n: 20
TRUST++
150
2004
0:01
2.33
0/337
8:23
8.029
4.055
0.59
0.005
0.131
8.035
0:277
0/04
3.51
0/05
110
tuned
3.3922
TRUST++
10.000
R.000)
0.000
R.0031
16,000
R.002
16 01.00
5.000
0.69
10.001
0.003
0100
10.001
6341
0L0041
13 08893
03 0.331
(6,001
8.990
Methods
TRUST
100001
50000
8.002
9312
16.0021
50.003
0.602
00000
882
0L003
833
(0006)
0.990
boosting
0.018
0.00
8.07
8.89
1880
MC
0.018
3.97
8.04
3.10
4.90
0.59
0.000
0.15
0.011
8.27
8.21
0/06
0.6981
0/03
1.24
asymptotic-
18.800
01:081
01000)
11:00
18.0811
0:000
14:000
10.0001
0.000
0.00
00.041
0.001
01801
00641
18800
0888)
(6655)
10.00
n: 50
n: 100
TRUST++
2004
0:000
2.221
0.545
2.544
3.54
3.24
tuned
TRUST+
MV
8.827
8:02
2013
01:11
8809
01961
19
8.84
83811
2014
Mothods
TRUST
10.0011
R1003
01000
13 9910
2011
21000
01.031
08000
0800
01/001
0.000
baosting-
GOME
5.055
0.043
6.25
8.022
0.066
6/26
MC
6.07
0.055
3.825
6.004
6.045
3.346
6.42
6.005
0.322
0.516
0.599
0.017
0.053
8.49
3.58
0.43
8 8.30
6005
3815
asymptotic
8900
R.000
R.000)
0.000
10.001
R.008
13,000
100000
8000
0000
10.000
3.000
500001
000001
880
0000
0.0041
13 13800)
0 880
0.000)
13 835
1dromat 801000
Idturmat Basgno
Idnommali B- Torno
Idnomatl: B 55000
grim B=1000
prem: 8 $t000
gmmni 100000
ummn Bes 15050
topnomall 1000
legnomat Be5080
legnomal: B 100100
Iegromaal B 15000
Idvomal B=2000
ddrormal 8-50000
Idnamas: B Loavo
Idrormal: B 55000
grnm: 8 8e50000
grmmi B 100000
grmmi B 1-15000
legnamat: Ba20o0
legnurmat Be
legromai B 10ose
lagrommel: B- 150e50
prom: Ba
Benchmarks
Benchmarks
Figure 12: Best methods (lowest MAE) for E-value on tractable models. In this case, we do not see a specific method that shows a clear dominance as in Figure [1] However, it is possible to notice the poor performance of asymptotic, which is never among the best methods. Our methods and the boosting are consistent across the sctups.
indicates that the local neighborhood has become sufficiently vell-populated to provide reliable quantile estimates.
Conversely, the performance remains remarkably stable across all other configurationss suggesting that TRUST++ is robust to secondary architectural choices. In terms of computational efficiency, only the number of trees K demonstrates a lincar increase in runtime; while MAE remains consistently low. This supports our default hyperparameter selections, demonstrating that TRUST+ achieves convergence in its proximity measure relatively early and prioritizes the quality of the local partition over sheer ensemble volume.
A.7 Performance under higher dimensions
To evaluate the scalability of our framework across varying parameter dimensions, we extend the Gamma GLM analysis presented in Section 5.3 Unlike the carlier nuisance parameter focus, we now consider the joint parametric space (e, 3, .i....,.. ,Bp), where p represents the increasing number of covariates.
Our benchmarking maintains a simulation budget of B = 10000 and utilizes the same default hyperpa- rameters for TRUST and the TRUST++-MV variant established in the muisance experiments. We assess the performance of all methods across 30 experiment repetitions for dimensions E (5, 10, 50). To ensure simulation stability in higher dimensions, we adapt the priors as follows:
For p E (5, We utilize the priors from Section 5.3.2 with 3 N(004) x 1(0,1)P-1 and Truncated Exponential 1.75).
For p = 50: B ~ N(0 0.0625) x (0, 0.01)49 and Φ ~ Truncated Exponential(1,1.33 1.5).
The comparative performance across these dimensions, illustrated in Figure I6 reveals a strong contrast
between our data-adaptive partitions and traditional benchmarks. While Monte Carlo (MC) approaches
33

---

Published in Transactions on Machine Learning Research (04/2026)
TRUST+ tuned
TRUST++ HV
Methioks
TRUST
boosting
MC
rfow
n: 10
20
TRUST+ tuned
Ba
TRUST++ MV
2:00
Macheds
TRUST
20 CaNIAA
Lat
boosting
MC
rflow
to und can
aodd can cran aoe
wes jusk Lovo
tos motors Hiloo
aie Reddboo
mesoral fi- 17880
two noom wensrrng 121p0
welisbeng 30000
verisbeng 2200gg
weticieng E- Lakeg
reght Hhap
reg --ybo
tramable Ki 10000
tramaaaa R Nooo
are 2000o)
reteove L200g
two meceosi Bi- 15650
mover 20000
maypri 10h00
eessdierp X-- I-1
eastierg cestberg iniz0iicg
eggi onisttrgg kelt00gp 10000
re ngt 20000000
20000
ngt Biihhpp
trectable 1000p
trectable 100g
tredable 2a000
tamabee Bie (lnop
Benchmarks
Benchmarks
Figure 13: Best methods (lowest MAE) for E-value on intractable models. As on BFF, TRUST+ tuned, TRUST++ MV and boosting the most consistent competitors. However, for this statistic, TRUST stands out in some scenarios, especially when n = 5. 10 (i.c., not in the extreme cases). As in the BFF case, nflow is competitive for the SIR benchmark with n = 1,5, 10.
fail to scale-becoming computationally prohibitive and eventually failing to run at p = 50 due to memory constraints imposed by the massive grid size both TRUST and TRUST+ maintain consistently low coverage MAE as dimensionality increases.
Furthermore, while asymptotic approximations exhibit a clear degradation in performance as p grows, our tree-based methods effectively mitigate the "curse of dimensionality" by focusing on the most informative regions of the parameter space. This targeted partitioning keeps the MAE nearly flat between p 10 and p = 50. Ultimately, both TRUST variants remain competitive with the boosting- alternative across the entire spectrum, demonstrating that our local neighborhood logic remains valid and robust even in higl-dimensional settings where traditional approximations falter.
B Algorithm for tuning M
In Algorithm
3
we specify the procedure for tuning M by using an available validation grid.
C Proofs
C.1 Section
2
Methodology
Proof of Theorem
Notice that
P e I(x)0 R(x)) = (7(x, C ((x, S &Co)
=P (h < Col Co) 8/2,
which proves the first statement. The proof for the second statement. is analogous.
34

---

Published in Transactions on Machine Learning Research (04/2026)
TRUST+ tuned
TRUST HV
Methioks
TRUST
boosting
MC
rfoo
n: 10
20
TRUST+ tuned
250 Cat
TRUST++
200
MV
Macheds
TRUST
Lat Ba
boosting
MC BI Baa
EAn bate bara baa car
wes Loog
tos motords Hille
air Reddboo
momoii viphiig
two nyoin a02otss
wessring Bim 12/0p)
weiibeng -200gg
weticheng E- (20eg
Bi 1000
regi. tho0
regh p-reoug
rig --tboe
tramub Kie L0bup
dramataa Plod
ar tim 13h00
reteovr L200g
two meceosi Bi- 15650
mover 20000
marpes 10h00
eesslierp y-- E-1
easthorg costeerg: Aniziticg
eggi onistarg kest000p
re 20000
10000
ngt Biihig
20-00000
trectable Do000
trect trecrable 1o0g
trecablee (a000
tarabee Bin 2000
welisbeng
Benchmarks
Benchmarks
Figure 14: Best methods (lowest MAE) for Waldo on intractable models. Similarly to BFF, TRUST++ tuned, TRUST++ MV and boosting are the methods that stand out the most. MC and nflow only appear the winners once each in 80 experiments, and TRUST has modest performance, with good results for the SIR benchmark with = 10.
Algorithm 3: Tuning algorithm for M
Data: Validation grid size Brune; grid of M's between O to K Mgrid; fitted TRUST++; number of simulated statistics nai in each grid point
Result: Tuned value of M, Mtuned simulate Ogrid = O0,....,,B.... from the prior;
for Moand E Mgrid do compute Co in TRUST++ with M fixed as Meand for each e O :
compute cover, (Č,0) 0) for each 0 E O grii (as in Eq. 10);
M MAEMann + M MAE(C, a) (as in Eq. 11
end
Miuned <- arg min {MAF
E
return Mtuned
C.2 Section 3 Theoreticcll Results
To summarize, in TRUST, the partition is created by building a regression tree that uses as the input and T(x, f) as the output. This tree, trained on the simulated data
(01, (x1, (0p, 7 (Xp,
naturally induces a partition on O
For TRUST++, we extend this approach by generating K regression trees. We then use Breiman's proximity measure p(0', f), which counts the number of times 6' and 0 appear together in the same leaf across these K trees. Finally, we define the partition A based on this proximity measure, using the equivalence relation
35

---

Published in Transactions on Machine Learning Research (04/2026)
depth
depth
0.030
0.025
0.020
1
0.015
10
15
20
min samples leaf MAE
min samples leaf Time
0.030
Time (3)
MAE
200
400
600
800
1000
1200
0
200
400
600
800
1000
1200
esstimttors
estimators
0.030
0.025
3-
0.020
0.015
200 Hyperparameter values
600
200 400
600
Hyperparameter values
Figure 15: Sensitivity analysis of TRUST+ the two moons task (B 15000).The left column shows coverage MAE; the right column shows runtime (s). Each row varies one hyperparamcter while others remain at defaults (K 200, M = K/2, leaf size = 300). Results represent. 30 repetitions with standard deviation error bars. Performance is primarily driven by sii__sample__aae _eles; once leaves are sufficiently populated (xe 300), coverage remains stable and invariant to tree depth or ensemble size, though the latter increases runtime linearly.
0~ o' 0) K; in other words, 0 and 0' must appear logether in the same leaves across all K trees.
This framework ensures that the partition A meets the necessary structural conditions to apply conformal prediction as stated in Theorem
Proof of Theorem This is straightforward application of conformal prediction (see Angelopoulos et al. The complete proof can found detail in (Cabezas et al. 2025b Theorem 2).
2023
To prove Theorem 3 we need the following lemma.
Lemma Let H(t ( S t(0) and A be a measurable set. Suppose p(x, 0) is the joint PDF of (X, f), satisfying p(z, 0) dr r(0) 0. Then we have
B|H(10) E A P(r(X,0) 110 E A).
36

---

Published in Transactions on Machine Learning Research (04/2026)
Mean MAE vs Dimension
0.150
Methods
Asymptotic
0.125
Boosting
Coverage MAE
MC
0.100
TRUST
0.075
+RUST++ MV
0.050
0.025
0.000
10
20
30
40
50
Dimension
Figure 16: Scalability across dimensions p {5, 10, 50)- Comparison of coverage MAF for the Gamma GLM joint parameter space. While Monte Carlo (MC) becomes computationally prohibitive and fails at 50, and asymptotic methods show degrading accuracy, both TRUST and TRUST++ remain robust with consistently low MAE.
Proof. Define = P(0 E A), the result follows from a straightforward calculation, as shown next.
E [((t))) E A|
E{H(t()){{ E A]]
P(0 E A)
=c" f H(t]0)I{0 e Alr(0)d0
=c- / /P(C(X, 0) & 40)T(O E A)r(O)d0
x-) / ((A)
[(Ht0) / [[7 0) s1p(,, 0)dr) T(O E A)r(0)a0
- / [[[(x, 0) s $|u[[0 Alp(2, 0)dzd0 -c-IP(r(X,0) 0) 51,0 A) = -P((XX, 0) s 110 E A).
Under the Assumption 1 for a sufficiently large B, we achieve H(10')= 2 (n(0''), for any 0' e and E IR. Based on this approximation, outline here the intuition for how Theorem 3 will be established in detail later.
Specifically, for a fixed 0 of interest, by Lemma
P(r(XX 0') S t 0' E A(0)) E(H(4)) e A(0))
((A 0' e A(8))
where the first equality results from straightforward calculation with its proof provided in the appendix.
37

---

Published in Transactions on Machine Learning Research (04/2026)
By the construction of the partitions in TRUST and TRUST++ if E A(f), then Hz(()0') (t(0))) Therefore,
P(r(X, $110' E A(O)) ~ (()) 1 A(O))
[[HH((t(0) e A(0))
[[ A(0)
H(O0).
This implies that, as long as our approximation HB closely matches H, our partition-based estimate will serve a reliable approximation of the test statistics distribution. In particular, as discussed in Section 2.1.1 let Co, H (alf) denote the adjusted a-quantile of the values (T(X,, 0.) b IA(o)). Then,
P (8 ( = HC, (0) = P(T(X, 0) S Co,al 6 E A(0)) a,
suggesting that our methods indeed should achieve optimal coverage.
Remark. As described in Section in practice may use relaxed neighborhoods controlled by M. When M < K, these neighborhoods need not induce a partition of O. In contrast, the asymptotic result proved here is stated under the partitioning regime M = K. This assumption is natural when letting the simulation budget grow (B + oo): with sufficiently many simulations, each cell populated by enough points so that the resulting partition non-degenerate. Accordingly, the proof of Theorem 3 bolow carriod out in the partitioning case M K.
Proof of Theorem Fix 0. By Assumption for any 6, > 0, there exists Bo and a subset C (0 X)
such that IP(T)) 8 and, conditionally on
(t H(()0) Se.
sup İER
By Lemma I
P(r(X, 110' E A(O)) B(H(10)) A(0)).
Conditionally on r, we know that
H(t 2 HB((t(') -
holds uniformly for any possible value of 0' E 0, R. Thus,
P(T((, 0) 1/00 e A(0), ([H(L)))) E T()),
> E[ = A(0),1]
= E. E[ A(), -E.
Note that, in the partitioning regime M = K, the collection (A(()) forms partition of θ. Thus, if 0' e A()), then 0 and 0' belong the same cell, implying A(8') A(0) and consequently HB(t 0') A 0). Therefore,
P 0') < 1100 E A(0),T) 2 E[ A(0), - =E(A A(0), r] -E.
Using the fact that, conditionally on r,
An(110) 2 H(U0) e.
we obtain
P(r(X, 8') & t/0' A(0), 2 ((HT 10 A(0)
E[H(t(0)| A(0),1] 2E.
38

---

Published in Transactions on Machine Learning Research (04/2026)
Since (t(0) is constant, it follows that
P(r(X, 0) < t/0' E A(0).r) 2 ([H(t() 0 A(0)) -2
=[(10)B[1] 0' E A(O),T) -28 -(40)) 2
Thus, for any t E R:
P((((X,0) S 1/00 E A(O)) 2 P(r(X, S 1/0' E A(0),
2 (1 - (1) 2s)
H( - 6H(1 2(1 - 8) + 20c H(0)) -82.
2 H(0))
In similar manner, we can show that
P(r(X,0) 5 1/0 E A()),T) S (t + 28.
Assumption 3), we assume there exists 0 < y < 0.5 such that P(Ø' A())) BB. This assumption reasonable, as implies that the partitions (i.e., the leaves) gener- ated by the tree-based estimators are uniformly well-populated. Tn other words, each leaf contains sufficient number of obscrvations, ensuring that the coverage propertics hold consistently across the partitions. There- fore,
Following [Meinshausen & Ridgeway 2006
5
0()(X,0) t10 E A(O)) & P(T( S t/0' e (()) +
BB
< F s t/0 e A) +8 H(10)) 2 +
Thus, for any e, and any R:
IF(((X,) A())) (+ 2$ + 6.
Taking t = Co a), by the construction of the empirical quantile over A(0),
[(1 a) -H(CC,, - - a)10) s 28 5,
which is equivalent to Since we can make e and d arbitrarily small by choosing B large enough, we conclude that
[1 (1-o) a) - P (BE Rx(X))0) s 2e 6.
lim P (0 e (x)() = - a. 8-500
39


Reviews:
[{"review_id": "GLRc44L4Zl", "reviewer": "Reviewer_jmdt", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "Yes", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "The paper introduces TRUST and TRUST++, two methods for calibrating statistical confidence sets using simulated data. The key idea for the two methods respectively is to partition the parameter space either via regression trees (TRUST) or random forests (TRUST++), then locally calibrate cutoffs using conformal ideas on simulated data. The main theoretical result (Theorem 3) establishes \"B\"-asymptotic (i.e., in terms of the number of simulations) conditional coverage as the simulation budget grows, which is interestingly independent of the observed sample size n. Decent experimentation was done across tractable likelihoods, LFI benchmarks, and nuisance-parameter settings show tuned TRUST++ matching or outperforming boosting, Monte Carlo, and asymptotic baselines, particularly in small-n regimes.", "claims_and_evidence_yn": "Yes", "claims_explanation": "This paper was well-written and well-executed.\n\nThree main claims:\n(1) Probably the main claim \"TRUST/TRUST++ achieve distribution-free finite-sample local coverage and $B$-asymptotic conditional coverage\" - with the main theorem for this being theorem 3. The result seems interesting, and the proof and justification looks decent to my eyes. But perhaps one thing to flag, the theorem 3 is for the strict $M = K$ variant (which induces a proper partition), whereas the generalised version with tuned $M < K$ does not always control coverage. I think the authors note this themselves on page 8 \"does not always control coverage for M < K (Guan, 2023), since it does not induce a partition, in practice it has good performance\", I think perhaps - if TRUST++ does not actually meet the main assumption for theorem 3 - it could be mentioned/highlighted more clearly in the paper. That said - I am not fussy on this and am happy with decent empirical results.\n\n\n(2) Strong empricial results for TRUST and TRUST++. The turned TRUST++ method outperforms existing alternatives, particularly in small-n scenarios, and the comparisons seem fair. That said, the bar charts (Figures 4-5) perhaps overstate the practical advantage, and the appendix heatmaps reveal that MAE improvements over boosting are somewhat modest (but still non-negligible). The \"frequency with which each method achieved the best performance\" plots don't really convey this nuance. Personally, I found the appendix plots much nicer for assessing the experiments and largely ignored figures 4-5. \n\n\n(3) Nuisance parameters and uncertainty quantification. No issue here to my eyes.", "audience_yn": "Yes", "audience_explanation": "Yes. The paper should interest SBI/LFI practitioners directly and researchers interested in conformal methods. I personally found the findings interesting.", "requested_changes": "Again I think this a fairly strong paper suitable for TMLR.\n\nFollowing are largely optional at author's discretion:\n\n1. Clarify the theory-practice gap for TRUST++ (tuned): as noted before, a short remark perhaps in Section 2.2 or 3.2 explicitly stating that Theorem 3's guarantee applies to the $M = K$ partition variant, and that the tuned version operates outside this, would help. Perhaps just one sentence somewhere to make this more clear to the reader (if it really does violate the assumption for theorem 3).\n\n2. Supplement win-count figures: figures 4–5 lose information about the magnitude of MAE differences. Personally, I'd prefer a complementary figure closer to the appendix heatmaps in the main text. But If other reviewers are fine with the win counts as-is I won't insist - and can be done at your discretion.\n\n3. Minor typo: on page 16, \"folllowing\" -> \"following\".", "broader_impact_concerns": "N/A", "additional_comments": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "G2WeMKSysH", "reviewer": "Reviewer_sUbn", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "Yes", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "This paper addresses a critical intersection in modern statistics: the inability of traditional methods to construct valid confidence sets when sample sizes are small or when likelihood functions are intractable (Likelihood-Free Inference or LFI). The authors propose two novel methods, TRUST and TRUST++, which adapt conformal prediction techniques to statistical inference. These methods utilize simulated data to calibrate confidence sets, ensuring they achieve the correct coverage levels (e.g., 95%) even in complex, non-asymptotic settings.\n\nStandard frequentist confidence sets rely on asymptotic theory (large sample sizes) or tractable likelihoods. In modern science (e.g., high-energy physics, epidemiology), models often define a simulator rather than a clear likelihood function.\n\n1. Research Challenge: When $n$ (sample size) is small, or the model is complex, traditional confidence sets often under-cover or over-cover the true parameter $\\theta$.\n\n2. Research Goal: Construct a confidence set $R(X)$ such that $P(\\theta \\in R(X)|\\theta) = 1-\\alpha$ (conditional coverage) without relying on asymptotics.\n\nThe paper introduces a \"distribution-free\" calibration approach that bridges conformal prediction and parameter inference.\n\n1. TRUST: A partition-based method that estimates the distribution of test statistics (like the Bayes Factor or E-value) using a tree structure.\n\n2. TRUST++: An enhancement of TRUST that utilizes random forests (ensembles of trees) to reduce variance and improve the estimation of the critical values required for calibration.\n\nBoth methods rely on having access to a simulator (generative model) to generate training data for calibration. The paper claims to provide both finite-sample local coverage and asymptotic conditional coverage. This is significant because most LFI methods only offer approximate guarantees.", "claims_and_evidence_yn": "Yes", "claims_explanation": "The authors tested the methods on both Tractable (Normal, GMM) and Intractable (SLCP, Two Moons, SIR, Weinberg) models.\n1. Performance: TRUST++ (specifically the \"tuned\" and \"MV\" variants) consistently outperformed the baseline methods (Naive Monte Carlo and Asymptotics).\n2. Robustness: In intractable likelihood settings (LFI), the naive MC method performed poorly, whereas TRUST++ maintained coverage close to the nominal level.\n3. Efficiency: The \"Boosting\" baseline was competitive in some areas, but TRUST++ generally provided the most robust results across different sample sizes.\n\nThe methods provide uncertainty quantification for the estimated confidence sets, allowing users to know if they need more simulations to trust the result. It offers a practical solution for scientists working with \"Black Box\" simulators who need rigorous error bars (confidence sets) but cannot rely on standard textbook formulas due to small data limits.", "audience_yn": "Yes", "audience_explanation": "Statistical inference should be an important part of machine learning, while conformal prediction is one of the most interesting problems in this area.", "requested_changes": "1. Like many LFI methods, this approach relies on the ability to generate large amounts of simulated data ($B = 30,000$ in their experiments). If the simulator is extremely expensive computationally (e.g., complex fluid dynamics), this might be a bottleneck. 3. The author claimed this method as  \"data efficient\" (requires few real observations) but \"simulation expensive\".\n\n\n2. While they demonstrate success on multi-dimensional parameters (e.g., 5D in SLCP), conformal methods can sometimes struggle with the \"curse of dimensionality\" regarding the efficiency of the partition space, though the use of Forests (TRUST++) attempts to mitigate this.\n\n\n3. The experiments are restricted to relatively low-dimensional parameter spaces. The most complex model shown (SLCP) has only 5 parameters ($\\theta \\in \\mathbb{R}^5$). Partition-based methods (like the decision trees used in TRUST) and even random forests are notoriously susceptible to the \"curse of dimensionality\". As the dimension of $\\theta$ increases, the number of partitions required to accurately estimate the distribution grows exponentially. It is unclear if TRUST++ would maintain its performance or coverage guarantees for models with 50, 100, or more parameters, which are common in modern machine learning-based inference.\n\n4. The baselines might be considered \"weak\" or \"standard\" rather than state-of-the-art, such as naive Monte Carlo. The paper does not appear to compare against other modern neural density estimation techniques (like Normalizing Flows used directly for calibration) or other recent conformal inference methods specifically designed for regression/conditional density estimation (e.g., CQR, DCP) adapted for this setting.", "broader_impact_concerns": "May promote the research in AI4Science area.", "additional_comments": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "ipo1KYW8fk", "reviewer": "Reviewer_nnB2", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "Yes", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "The paper proposes TRUST and TRUST++, two tree-based methods for estimating the distribution of a test statistic in likelihood-free settings in order to construct confidence sets with valid conditional coverage. TRUST partitions the parameter space using a regression tree, while TRUST++ uses bagged trees to obtain smoother, more accurate local neighborhoods for estimating quantiles. The paper also extends the methodology to handle nuisance parameters and provides uncertainty quantification for the resulting confidence sets. The authors provide theoretical guarantees for finite-sample and asymptotic coverage and show strong empirical performance across simulated and real-world statistical models.", "claims_and_evidence_yn": "Yes", "claims_explanation": "The paper’s main claims are generally supported by the material provided. The authors give clear definitions of the problem, explain how TRUST and TRUST++ are constructed, and provide theoretical arguments for the coverage properties they aim to achieve. The experiments are reasonably designed and illustrate that the methods behave as claimed in several settings. Overall, the combination of theory and empirical results is sufficient to support the main points of the submission.", "audience_yn": "Yes", "audience_explanation": "The paper tackles a problem that appears in many simulation-based or likelihood-free inference settings, and the proposed methods would likely interest researchers working on conformal prediction, uncertainty quantification, or approximate Bayesian computation. The tree-based approach and the handling of nuisance parameters are practical ideas that could be useful to others in the community.", "requested_changes": "1- Some steps in the construction of TRUST and TRUST++, especially the motivation for the partitions and the role of $\\rho$, are difficult to follow on first reading. A bit more intuition or a small running example would help readers understand the flow of the algorithm.\n\n2- The current presentation nuisance-parameter is technically correct but hard to digest. It would help if the authors added a short summary of the key idea and gave a simple illustration of how the minimization over ν becomes tractable.\n\n3- Since TRUST++ relies on bagging and proximity, a clearer explanation of how it differs from a regular RF-based estimator would strengthen the contribution.\n\n4- A bit more information on simulation settings (how the $\\theta$ values were chosen, how sensitive results are to tree depth or number of trees) would improve the reproducibility of the experiments.\n\n5- Both TRUST and TRUST++ rely on tree depth, leaf size, and number of trees. The paper briefly mentions these choices, but does not fully explore how sensitive the conclusions are to these hyperparameters.", "broader_impact_concerns": "N/A", "additional_comments": "", "extra_scores": {}, "extra_text": {}}]

Author rebuttal:

Decision: Accept (poster)