Non-Asymptotic Uncertainty Quantification in High-Dimensional Learning
Authors: Frederik Hoppe
Claudio Mayrink Verdun
Hannah Laus
Felix Krahmer
Holger Rauhut
Venue: NeurIPS 2024 spotlight (2024)
Keywords: high-dimensional regression
uncertainty quantification
model-based deep learning
debiased estimator
inverse problems
Primary area: probabilistic_methods
TL;DR: Our paper derives and validates a data-driven approach to construct non-asymptotic confidence intervals for high-dimensional regression that overcomes the issues faced by previous asymptotic uncertainty quantification techniques.

Abstract: Uncertainty quantification (UQ) is a crucial but challenging task in many high-dimensional learning problems to increase the confidence of a given predictor. We develop a new data-driven approach for UQ in regression that applies both to classical optimization approaches such as the LASSO as well as to neural networks. One of the most notable UQ techniques is the debiased LASSO, which modifies the LASSO to allow for the construction of asymptotic confidence intervals by decomposing the estimation error into a Gaussian and an asymptotically vanishing bias component. However, in real-world problems with finite-dimensional data, the bias term is often too significant to disregard, resulting in overly narrow confidence intervals. Our work rigorously addresses this issue and derives a data-driven adjustment that corrects the confidence intervals for a large class of predictors by estimating the means and variances of the bias terms from training data, exploiting high-dimensional concentration phenomena. This gives rise to non-asymptotic confidence intervals, which can help avoid overestimating certainty in critical applications such as MRI diagnosis. Importantly, our analysis extends beyond sparse regression to data-driven predictors like neural networks, enhancing the reliability of model-based deep learning. Our findings bridge the gap between established theory and the practical applicability of such methods.

Non-Asymptotic Uncertainty Quantification in High-Dimensional Learning
Frederik Hoppe
Claudio Mayrink Verdun"
RWTH Aachen University hoppedmathc. marth-aachen. de
Harvard University claudioverdunsseas www.aar...aa.a.. edu
Hannah Laus"
Felix Krahmer TU Munich & MCML felix. krahmer de
Holger Rauhut
TU Munich & MCML hannah. lausStum... de
LMU Munich & MCML
Sauhut@matle 1mu, de
Abstract
Uncertainty quantification (UQ) crucial but challenging task in many high- dimensional learning problems increase the confidence of given predictor. We develop a new data-driven approach for UQ in regression that applies both to classical optimization approaches such as the LASSO well as to neural networks. One of the most notable UQ techniques is the debiased LASSO, which modifies the LASSO allow for the construction of asymptotic confidence intervals by decomposing the estimation error into a Gaussian and an asymptotically vanishing data, the
bias component. However, in real-world problems with bias term often too significant to disregard, resulting in overly narrow confidence intervals. Our work rigorously addresses this issue and derives a data-driven adjustment that corrects the confidence intervals for large class of predictors by estimating the means and variances of the bias terms from training data, exploiting nigh-dimensional concentration phenomena. This gives rise to non-asymptotic confidence intervals, which can help avoid overestimating certainty in critical applications such as MRI diagnosis. Importantly, our analysis extends beyond sparse regression to data-driven predictors like neural networks, enhancing the reliability of model- based deep learning. Our findings bridge the gap between established theory and the practical applicability such methods.
Introduction
The past few years have witnessed remarkable advances in high-dimensional statistical models, inverse problems, and learning methods for solving them. In particular, we have seen surge of new methodologies and algorithms that have revolutionized our ability to extract insights from complex, high-dimensional data 3T Also, the theoretical underpinnings of the techniques these fields have achieved tremendous success. However, the development of rigorous methods for quantifying the uncertainty associated with their estimates and underlying parameters, such constructing confidence intervals for given solution, has lagged behind, with much of the underlying theory remaining elusive.
In high-dimensional statistics, for example, even for classical regularized estimators such as the LASSO [4-6), was shown that closed-form characterization of the probability distribution of the estimator in simple terms is not possible, e.g., [ Theorem 5.1]. This, in turn, implies that is very challenging establish rigorous confidence intervals that would quantify the uncertainty of
Equal contribution. Correspondence to hoppe@matth..rthh.aeee
38th Conference on Neural Information Processing Systems (NcurIPS 2024).

---

<1
415
201
(a) w/o data adjustment
(b) w/ Gaussian adjustment
(c) w/ data adjustment
340
181
(d) coverage over all components
(c) coverage on the support
Figure 1: Illustration of the confidence interval correction. Figs. Ta 1b 1c show the construction
of CIs with standard debiased techniques (w/o data adjustment) and with our proposed method (w/ Gaussian adjustment- Thm. 3 in Fig. Ib and data adjustment Thm. 2 in Fig. Ic respectively.
The red points represent the entries that are not captured by the Cls. Additionally, Fig. ld shows box plots of coverage
all components, and Fig. shlssows them on the support, non-zero pixels. In the last two plots, the left box refers to the asymptotic and the right to the non- asymmtotii CIs based on Gaussian adjustment of 500 feature vectors. We solve sparse regression problem via the LASSO, where A E 4000x 10000 I E C is 200-sparse, and the noise level is = 10%. The averaged coverage over 250 vectors with significance level a 0.05 of the asymptotic confidence intervals is W (0.05) 0.9353 and on the support 15 (0.05) = 0.8941. Confidence intervals built with our proposed method yield for Gaussian adjustment ht (0.05) = 0.9684 and on the support hg (0.05)= = 0.9421, and for data-driven adjustment h(0.05) hg (0.05) = 1. For more details, cf. Section 5. and Appendix
Azz +
proposed and analyzed the debiased LASSO, also known as the desparsified LASSO, a procedure to fix the bias introduced by the penalty in the LASSO: see (9] Corollary 11] and [ for discussion on the bias induced by the L regularizer. The debiased estimator derived in the aforementioned works has established principled framework for obtaining sharp confidence intervals for the LASSO. initiating a statistical inference approach with UQ guarantees for nigh-dimensional regression problems where the number of predictors significantly exceeds the number of observations. Recently, this estimator was also extended in several directions beyond f f1-minimization which include, for example, deep unrolled algorithms [22 and it has been applied many fields like magnetic resonance imaging both with classical high-dimensional regression techniques as well as recent learning ones [12 [4];
such estimated parameters. To overcome this, series of papers 8-10
see the
paragraph related works Section below.
The idea of the debiased LASSO is that its estimation error, i.e., the difference between the debiased estimator and the ground truth, can be decomposed into Gaussian and remainder/bias component. It has been shown in certain cases that the foo norm of the remainder component vanishes with high probability, assuming an asymptotic setting, i.c., when the dimensions of the underlying model grow within a specific rate, see [10 (5) for details. In this case, the estimator is proven to be approxinately Gaussian from which the confidence intervals are derived. However, practice, one needs to in very high- -igh-iinensional regime with enough data for these assumptions to kick in. In many applications with a finite set of observations, the remainder term does not vanish; can rather substantially large, and the confidence intervals constructed solely based on the Gaussian component fail account for the entire estimation error. Consequently, the derived confidence intervals are narrower, resulting in overestimation of certainty. This issue is particularly problematic in applications where it is

---

crucial to estimate the magnitude of a vector coefficient with high degree of confidence, such as in medical imaging applications.
Moreover, according to the standard theory of debiased estimators, the estimation of how small the remainer term is depends on how well one can quantify the &z and & bounds for the corresponding biased estimator, c.g., the LASSO [5]. Although sharp oracle inequalities exist for such classical regression estimators, cf. related works, the same cannot be said about when unrolled algorithms are employed. For the latter, generalization bounds are usually not sharp or do not exist.
In this paper, we tackle the challenge of constructing valid confidence intervals around debiased estimators for the parameters in high-dimensional learning. The key difficulty lies in accounting for the remainder term in the estimation error decomposition see Equation B which hinders the development of finite-sample confidence intervals. We propose a novel non-asymptotic theory that explicitly characterizes the remainder term, enabling us to construct reliable confidence intervals in the finite-sample regime. Furthermore, we extend our framework to quantify uncertainty for the output of model-based neural networks, which, in turn, are used to solve inverse problems. This paves the way for a rigorous theory of data-driven UQ for modern deep learning techniques. We state an informal version of our main result, discussed in detail in Section 3
Theorem (Informal Version). Let x01), I(() E CN be i.i.d. data. Let b(0) = Axx()) + e(0 be
high--dimensional regression model with noise g(i) CN INx)). With the data, derive, for
significance level a, a confidence radius ra(a) for new sample's 's component x; ((+1), Let (2")
be the debiased estimator based on a (learned) high--dimessonal regression estimator 2/7 Then, it holds that
P - -
S <r((a)) 21
Theorem I has far-reaching implications that transcend the classical regularized high-dimensional regression setting. For example, it enables the establishment of rigorous confidence intervals for learning algorithms such as unrolled networks [66. To our knowledge, obtaining rigorous UQ results for neural networks without relying on non-scalable Monte Carlo methods remains challenging problem [17). To address this and quantify uncertainty, our approach combines model--ased prior knowledge with data-driven statistical techniques. The model-based component harnesses the Gaussian distribution of the noise to quantify the uncertainty arising from the noisy data itself. We note that the Gaussian assumption for the noise is not limitation, and extensions non-Gaussian distributions are also possible via, e.g., a Central Limit Theorem-type argument, as clarified by [T]. We make a Gaussian noise assumption here for the sake of clarity. Complementing this, the data-driven component is imperative for quantifying the uncertainty inherent in the estimator's s performance. Moreover, our approach does not require any assumptions regarding the convergence or quality properties of the estimator. This flexibility enables the debiased method to apply to a wide range of estimators.
Contributions. The key contributions in this work are threefold
1. We solve the problem illustrated in Fig. by developing a non-asymptotic theory for constructing confidence intervals around the debiased LASSO estimator. Unlike existing approaches that rely asymptotic arguments and ignore the remainder term, our finite-sample analysis explicitly accounts for the remainder, clarifying an important theoretical gap and providing rigorous guarantees without appealing to asymptotic regimes.
2. We establish a general framework that extends the debiasing techniques to model-based deep learning approaches for high-dimensional regression. Our results enable the principled measurement of uncertainty for estimators learned by neural networks, a capability crucial for reliable decision-making making in safety-criticall applications. We test our approach with -fate-of-thh-arr unrolled networks such as the It-Net [18].
3. For real-world medical imaging tasks, we demonstrate that the remainder term in the debiased LASSO estimation error can be accurately modeled Gaussian distribution. Leveraging this finding, we derive Gaussian adjusted CIs that provide sharper un- certainty estimates than previous methods, enhancing the practical utility of debiased estimators in high-stakes medical domains.
'The code for our findings is available on GitHub:
dim_ learning
3

---

2 Background and Problem Formulation
In numerous real-world applications, we encounter high-dimensional regression problems where the number of features far exceeds the number of observations. This scenario, known as high-dimensional regression, arises when we aim estimate N features, described by x0 E CN from only few m target measurements b E Cmm. where m & N. Mathematically. this can be expressed as linear model Ar + E. where E C is the measurement matrix and additive Gaussian noise with variance o², In the presence of sparsity, where the feature vector x0 has only & non-zero entries (& < N), a popular approach to solve the LASSO, which gives an estimator i obtained by solving the following & rrreullarieed optimization problem:
C.N(O, o² INXN) is
min TEEN 2m
|Ar
bll2 + A||2//2-
However, the LASSO estimator is known to exhibit a systematic bias, and its distribution is intractable, posing challenges for uncertainty quantification [7]. To address this limitation, debiasing techniques have been developed in recent years g [0]. The debiased LASSO estimator, .", defined as:
1 (AE b),
m
where M is a correction matrix that could be chosen such that jE(,.. IMM - INNN)el
small. Here, Σ A'A, We refer 15 for more detailed description of how choose M. Remarkably, the estimation error
6" x0 MA'e/m (MS - INXN) #).
can be decomposed into Gaussian component W CN(O, eكX and a remainder term R that vanishes asymptotically with high probability [15] Theorem 3.8], assuming Gaussian measurement matrix A. Such result was extended to matrices associated a bounded orthonormal system like a subsampled Fourier matrix, allowing for extending the debiased LASSO to MRI [19]. The decomposition 3 and the asymptotic behavior of R enable the construction of asymptotically valid CIs for the debiased LASSO estimate, providing principled UQ for high-dimensional sparse regression problems.
However, in real-world applications involving finite data regimes, the remainder term can be signifi- cant, rendering the asymptotic confidence intervals imprecise or even misleading, illustrated in Fig. This issue is particularly pronounced in high-stakes domains like medical imaging, where reliable UQ is crucial for accurate diagnosis and treatment planning. Second, the debiasing techniques have thus far been restricted to estimators whose error is well quantifiable, leaving the challenge of how they would behave for deep learning architectures open. such cases, the behavior of the remainder term is largely unknown, precluding the direct application of existing debiasing methods and hindering the deployment of these methods sen-ssenitive applications.
prominent example for solving the LASSO problem in
① with an unrolled algorithm is the ISTA
20 21:
1 k+1 =
AA -INNN-LAA" 1
k
Here, > 0 is a step-size parameter, and Sx(x) is the soft-thresholding operator. The work 22 interpreted each ISTA iteration as layer of a recurrent neural network (RNN). The Learned ISTA (LISTA) approach learns the parameters Wtt, Wh.. x instead of using the fixed ISTA updates:
I k+1 Sx (W + W/b).
In this formulation. LISTA unrolls K iterations into K layers, with learaale parameters (W* X per layer. The parameters
leamedd by minimizing the reconstruction error minx,w Ela W, ((0) ((0)) - (0 on training data (z(*), (()). Unrolled neural networks like LISTA have shown promise as model-based deep learning solutions for inverse problems, lever- aging domain knowledge for improved performance. Such iterative end-to-end network schemes provide ofteeoff-th-aan reconstructions for inverse problems
[18]. Recently, the work [[2] proposes

---

framework based on the debiasing step to derive confidence intervals specifically for the unrolled LISTA estimator. However, similar to the previously mentioned debiased LASSO literature, it only handles the asymptotic setting. One of the main goals of this paper is to overcome such limitation of the current theory.
Related Works.
High-dimenssonal regression. High-dimensional regression and sparse recovery is now well- established theory, see 0 3 23 and references therein. In this context, several extensions of the LASSO have been proposed such as the elastic net [24), the group LASSO (2], the LASSO with nuclear norm penalization [26), the Sorted L-One Penalized Estimation (SLOPE) [27] which adapts the l1- f--norm to control the false discovery rate. In addition to convex penalty functions, concave penalties have been explored to address some limitations of the LASSO, e.g., the Smoothly Clipped
Absolute Deviation (SCAD) penalty (8 and the Minimax Concave Penalty (MCP) (29]. Non-convex
variants of the LASSO for f--norm (p < 1) minimization were also studied [30 31 as well as noise-
blind variants such as the square-root LASSO [32 [33] Scalable and fast algorithms for solving the
LASSO and its variants include semi-smooth Newton methods (34] and IRLS
35
LASSO theory. Given how ubiquitous and studied such an estimator is, it is difficult to do justice to all the papers that have contributed to such a theory. Several works have established oracle inequalities for the LASSO [3) Another key theoretical result is the consistency of the LASSO in terms of variable selection. [41 and [42] established the consistency of the LASSO while 43 analyzed the sparsity behavior of the LASSO when the design matrices satisfy the Restricted Isometry Property.
Debiased estimators. After the first papers about the debiased LASSO [+) some works have focused on improving its finite-sample performance and computational efficiency [15] 44 The size of the confidence intervals derived for the debiased LASSO has been proven to be sharp in the minimax sense [45). Debiased estimators have been extended in several directions, e.g..
47 Recently, 4S established asymptotic normality results for a debiased estimator of convex regularizers beyond the ( nnorm In the context of MR images, [49] explored debiased estimator for inverse problems with total variation regularizer. Debiased estimators have also been recently extended to unrolled estimators-
[9
discussion in the next paragraph in 12 13]
Algorithm unrolling and model-based deep learning for inverse problems. The idea of unfolding the iterative steps of classical algorithms into deep neural network architecture dates back to 22). which proposed the Learned ISTA (LISTA) to fast approximate the solution of sparse coding problems. Several works have extended and improved upon the original LISTA framework (50455)
(56] proposed the Learned Primal-Dual algorithm, unrolling the primal-dual hybrid gradient method
for tomographic reconstruction. [57] proposed the Deep Cascade of Convolutional Neural Networks (58) unfolded proximal gradient descent solvers to learn their parameters for ID TV regularized problems. (16) introduced general framework for algorithm unrolling. [59] developed MoDL, a model- based deep learning approach for MRI reconstruction that unrolls the ADMM algorithm. 160 proposed a proximal alternating direction network (PADNet) to unroll nonconvex optimization. See also the surveys for more information about unrolling and also the connection with physics-inspiree methods [6] 62). [8 [3) developed the It-Net, an unrolled proximal gradient descent scheme where the proximal operator is replaced by a U-Net. This scheme won the AAPM Challenge 2021 [64] whose goal was identify the state-of-the-art in solving the CT inverse problem with data-driven techniques. A generalization of the previous paradigm is the learning to optimize framework that develops an optimization method by training, i.e., learning from its performance on sample problem 665 66).
(DC-CNN) for dynamic MRI reconstruction.
Uncertainty Quantification. There have been few attempts to quantify uncertainty pixel level for unrolled networks used in imaging processing, c.g., [67. However, such approaches are based on Bayesian networks and MC dropout (68), which requires significant inference time paired with a loss of reconstruction performance since the dropout for UQ is a strong regularizer in the neural network. Unlike prior work, contribution focuses on scalable data-driven method that is easily implementable in the data reconstruction pipeline.
Another method that became popular in the last couple of years is conformal prediction 669
which addresses the problem of constructing prediction bands C. X + {subsets of B} for a given level a with the property that for a new i.i.d. pair (a,, b.), we get IP(bm41 Ön (1))+1) 2 1 a, where the pairs (a., b.) ~ P, 1,..,, m are i.i.d. feature and response pairs from a distribution
5

---

P on A x B and the probability is over all of available data b,), = 1..... m + 1. Such method assumes that the regression coefficient z, in the model b, (Q,, + e, is fixed. In contrast to that, the debiased LASSO produces confidence intervals for individual pixels 2j
(1+1) where
j 11,,... 1,....,, .N when a new set of data points b((-1) ..... 1 is given for fixed set of measurement vectors @i:.... amn Recently, a few papers developed conformal prediction- -basod uncertainty masks
for imaging tasks 771 72.. Unlike the former, our method provides computationally inexpensive confidence intervals for a new given test image. Unlike the latter, which constructs an interval-valuee function for each pixel of a new image that holds in expectation (see Equation 2 in (72), our method
comes with pixel-wise coverage guarantees for each new test image.
3 Data-Driven Confidence Intervals
We now introduce our data-driven approach to correct the CIs. Instead of deriving asymptotic CIs from the decomposition n" 00 W + R. by assuming that R asymptotically vanishes, we utilize data (b(), (=()) 1=1 along with concentration techniques to estimate the size of the bias component R. We continue to leverage the prior knowledge of the Gaussian component W while extending the CIs' applicability to broad class of estimators, including neural networks. Our method ís summarized in Algorithm[] [] where the data is used to estimate the radii of the Cls, and in Algorithm 2 which constructs the estimator around which the CIs are centered. The following main result proves the validity of our method.
Theorem 2. Let x: ....).... (()) E CN be i.i.d. complex random variables representing ground truth data drawn from an unknown distribution Q. Suppose, that e( CN o² I mxxm) noise in the high-dimensional models b(0) = (i + e(), where A E CmxN, and independent of the (00)'s 's. Let &= = - CN be a (learned) estimation function that maps the data 155) to 5:5), which functioon
|R le]( (MÉ - INxN (2(()) - x00)] for fixed A and M. For .N, we denote the true but unknown mean with μ4 = BBIRS"1 [] and the unknown variance j = -I...,,,,
is an estimate for z(0) Set
with (ok) == E[(IR - -1,) respectively. Let 5; - R251 be the unbiased sample mean estimator and (6);) 1 1=1
(1 the unbiased variance estimator. Let E (0, and Y3 (0, ta)- Furthermore, set the confidence regions for the sample x((+)) ~ Q in the model ((++1) (x(1+1) + c(++1) as Ca(a) {z E C 1(<))] (1+1) -2 S <r((a)} with radius
ry(a)= (MEMM))
1 Tiaa
12 1-1
log
ta (a) (o)-))))
a(aa = | 12 - -1 -1
(4)
Vmi
Then, it holds that
P
ECj(a)) 2 21-a. a
(11)
(5)
Theorem 2 presents a way to achieve conservative confidence intervals that are proven to be valid, i.e. are proven to contain the true parameter with a probability of 1 a. Its main advantage is that there are no assumptions on the distribution Q (except that 0% exists), making widely applicable. Hence, Theorem 2 includes the worst-case distribution showing way to quantify uncertainty even in such an ill-posed setting. Especially in medical imaging, such certainty guarantees are crucial for accurate diagnosis. The proof exploits the Gaussianity of the component W well as an empirical version of Chebyshev's inequality, which is tight when there is no information on the underlying distribution. The detailed proof can be found in Appendix B For a thorough discussion Theorem 2 including practical simplifications and the dependence of y on the confidence interval length, we refer to Appendix|A [A]
More certainty comes with the price of larger confidence intervals. If there is additional information on the distribution of Rt like the ability to be approximated by a Gaussian distribution, then the confidence intervals become tighter. This case, which includes relevant settings such as MRI, is discussed in Section

---

Algorithm Estimation of Confidence Radius
1: Input: Estimation function X, dictionary matrix A, correction matrix M, data points
(b()) (0)
significance level
2: for 1, 1=1" 1..1,,,,,,, do
3: Compute $(0, R E CN via ((1) x(b(0) and n2)") ez eME |NNN)(E0- -x(0).
4: end for
5: for
do
6: Estimate 8, - Σ R and (oh) - Σ (1Rg) - -,,3
a (MMM))
7: Solve ry(a) min
log
- ((1 7)a) (GRR) + S
7
8: end for
9: Output: Radii of confidence regions (ry(a)))
Algorithm 2 Construction of Confidence Regions
1: Input: Estimation function X, dictionary matrix A, correction matrix M, measurement b, radii (==(a)))== (derived from Algorithm | or Theorem
2: Compute estimator å = (b).
3: Construct debiased estimator via 2"
1 (b (b- A?) m
4: for j j= N do
5: Construct confidence region Ca = {= C jay - =| S rr(()}}
6: end for
7: Output: Debiased estimator x" and confidence regions (C)(a))
4 Confidence Intervals for Gaussian Remainders
Valid confidence intervals, i.c.. those with correct coverage probability, can be derived most straight- forwardly when the distribution of the remainder term known and easily characterized. In such cases, more informative distributional assumptions lead to potentially tighter confidence intervals compared to Theorem which makes assumptions about the remainder component. In this section, we derive non-asymptotic confidence intervals assuming the remainder term to be approximated by Gaussian distribution.
Theorem 3. Let 4" E C be debiased estimator for x E CN with a remainder term R CN En/m). Then, Cy(a) f C 2y| S ry())} with radius
g((a== (O2(MEM') 333 (En),;)
1 ④
log
(6)
m
valid, i.e. PP (zy E
For the proof, we refer to Appendix B We note, however, that the theorem can be generalized beyond the Gaussian case. particular, we present in Appendix Cla proof of this theorem for heavy-tailed distributions. In Appendix E we demonstrate empirically that the Gaussian assumption for the remainder term holds in wide range of relevant practical settings. This validation enables the application of the proposed confidence intervals derived under this assumption. These confidence intervals strike a careful balance between non- asymptotic reliability, ensuring valid coverage even in finite-sample regimes, and tightness, providing informative and precise uncertainty estimates. By leveraging the Gaussian approximation, which becomes increasingly accurate higher dimensions as illustrated in Figure our framework offers principled and computationally efficient approach to quantifying uncertainty in high-dimensional prediction problems. The variance of R can be estimated with the given data using, c.g., the unbiased estimator for the variance in Theorem 2
7

---

5 Numerical Experiments
We evaluate the performance of our non-asymptotic confidence intervals through extensive numerical experiments across two settings: (i.) the classical debiased LASSO framework to contrast our non- asymptotic confidence intervals against the asymptotic ones. (ii.) the learned framework where we employ learned estimators, specifically the U-net (73) as well as the It-Net [88. to reconstruct real-world MR images and quantify uncertainty. Our experiments demonstrate the importance of properly accounting for the remainder term in practical, non-asymptotic regimes. Each experiment follows the same structure:
1. Data Generation and Management: We fix the forward operator A and generate n > feature vectors (()))
1=1 and noise vectors e(6) (=1 with e(f ~ C.(00, mmxm). We obtain observations b(6) 2m1 via 8(0) Az( + c(t), We split the data (b(*), (b())=())) into an estimation dataset of size and a test dataset of size (1 + k = n). If we learn an estimator, we further split the data into training, estimation, and test sets.
2. Reconstruction: Depending on the experiment, obtain a reconstruction function X in one of the following ways: for the classical LASSO setting, we use the LASSO; for the learned estimator experiment, we train U-Net 73 or It-net [8] on the training data to serve as the reconstruction function X.
3. Estimation of Confidence Radii: We run Algorithm with A, N,M .M (that is chosen according to [15 ) the estimation data x())) /==
and predefined significance level a E (0, 1) to obtain radii ()( We optimize over y, therefore the 7y we use is the optimal one which leads to the smallest confidence intervals. To construct the final confidence intervals, the radii need to be centered according to the debiased estimator. For every new measurement we run Algorithm obtain tailored confidence intervals for the feature vector corresponding to In addition, we compute the CI for the Gaussian adjustment based on Theorem 3 using the estimation set to quantify the variance of R with the unbiased estimator for the variance as before.
4. Evaluation: We use the test dataset (0))),
1=1141 to evaluate adjustments. For each 600, we run Algorithm 2 to obtain confidence intervals C/P (a) |=ر for (1) We estimate PP(2))
(1) E C)(aa) by hy( = Ei 1 (af) ECj(a)) and average over all components
h(a) "E Since performance on the support S, the non-zero pixels, is crucial, we define the hit rate on S as hg TsT Σラ= (aft (a,) and average hs(a)
Eist4 h Note that the support may change with Moreover, do the same for the CI based on the Gaussian-ad radii.
5.1 UQ for Sparse Model-Based Regression
We consider setting aligned with existing debiased LASSO literature, e.g., [9] to demonstrate our approach's is extension of current UQ methods. The forward operator a complex Gaussian matrix A C= with dimensions N 10000, m 0.6N,, and Ag CO 1). We generate 750 O.1N-sparse features و(0ر) by randomly selecting m distinct indices from 1,...,N and drawing magnitudes from C. 1). With relative noise || [| ~ 0.2, we split the data (b(0) (()) >- into = 500 estimation and k 250 test data. For reconstruction, solve the LASSO (b) == argmin 1 Arr bll A||=|| with À 10 12 1 /2)log(N)) following 19..
m
With significance level 0.05, we run Algorithm | to obtain confidence radii, choosing M 15 and exploiting the relaxation
Averaged over the estimation data points, the & and || 0.9993 and I R = 1.1581. In existing literature, the foo norm is typically measured when the remainder term vanishes, as it is relevant for pixel-wise confidence intervals. Here, the remainder term is of comparable order as the Gaussian term and hence, too significant to neglect in confidence intervals derivation.
INxN
too norm ratios are: W
8

---

0.0
010
(a) w/o adjustment
(b) w/ Gaussian adjustment
(c) w/ data adjustment
Figure 2: Confidence intervals of asymptotic type 2a with Gaussian adjustment 2b and data-driven
adjustment 2c for one evaluation feature vector in the sparse regression setting described in Section
5.1
Evaluating it on the remaining l = 250 data points, the data-driven and Gaussian-adjusted averaged hit rates are ((0005) = 1, ht (0.05) = and h (0.05) = 0.9691, &g (0.05) = 0.8948, respectively. Neglecting the remainder term yields W (0.05) = 0.8692 and ng (0.05) = 0.6783, which is substantially lower and violates the specified 0.05 significance level. Fig. presents confidence intervals of each type for one data point (0)) detailed visualization of h (0.05). n ( (0.05), h (0.05), and ( is illustrated in Fig. and 4j the appendix. Further experiments with different sparse regression settings, including subsampled Fourier matrices, also presented in Appendix D
5.2 UQ for MRI Reconstruction with Unrolled Neural Networks
We extend the debiasing approach to model-based deep learning for MRI reconstruction using the U-Net and It-Net on single-coil knee images from the NYU fastMRI dataset 2 [74] 75). Here, the forward operator is the undersampled Fourier operator PF E C with N 320 x 320, m 0.6.N, the Fourier matrix F and a radial mask P, see Figure 5b The noise level a is chosen such that the relative noise is approximately 0.1. The data split into training (33370 slices), validation (5346 slices), estimation (1372 slices), and test (100 slices) datasets.
We then train It-Net [18 with 8 layers, combination of MS-SSIM 776 and lllsosees and Adam optimizer with learning rate 55 for 15 epochs to obtain our reconstruction function X.
With significance level 0.1, we run Algorithm
construct confidence radii, this time for the real part of the image instead of the magnitude, choosing M = I N n [15] and exploiting the relaxation
9 Averaged over the estimation data points, have [W I| 0.38 and W || 0.49, which indicates that the remainder term significant and cannot neglected. Evaluating the test data, the averages of the data-driven adjustment hit rates are ((0.1) )) = 0.9998, (0.10. = 0.9995 and the averages of the Gaussian adjusted hit rates are 1. = 0.9485, hg (0.1) = 0.9353, where the support refers to the largest 10% of the image pixels as the background cannot be clearly delineated numerically. Neglecting the remainder term, the hit rates of the asymptotic CIs are & (0.1) = 0.9306 and W (0.1) = 0.9152. As in the sparse regression setting, they are significantly lower. Fig. presents confidence intervals based on the data-driven adjustment and the asymptotic confidence intervals for region in one image (8) In addition, it contains box plot showing the distribution of the hit rates based on the Gaussian adjustment and the asymptotic hit rates. More experiments for UQ for MRI reconstruction can be found in Appendix D and Tables and 3
[7
We obtained data, which we used for conducting the experiments in this paper from the NYU fastMRI Initiative database (fastmri.mg (fastmri.med.nyu.u. med. 77 75). The data was only obtained from the NYU fastMRI investi- gators, but they did not contribute ideas, analysis, or writing to this paper. The list of the NYU fastMRI investigators, be found at fastmri.med med.ay..edu, it subject updates.

---

20
65
(a) ground truth
(b) w/ data adjustment
1.000
20-
0.755
0.925
687
60
(c) w/o data adjustment
(d) coverage over all components
Figure 3: Reconstruction obtained with the It-Net as described 5.2
Data-driven adjustment 90%
confidence intervals 3b and asymptotic 90% confidence intervals 3c for the region (50 pixels) in 320x320 knee image 3a Boxplots of hit rates 3d for 95% confidence level for the Gaussian adjusted and asymptotic confidence intervals.
Final Remarks
In this work, proposed data-driven uncertainty quantification method that derives non-asymptotic confidence intervals based on debiased estimators. Our approach corrects asymptotic confidence intervals by incorporating
estimate of the remainder component and has solid theoretical foun- dations. While the correction can be based on prior knowledge, e.g., a Gaussian distribution of the remainder term, we also derive CI based on data-driven adjustment without further information. This data-driven nature enhances its applicability to wide range of estimators, including model- based deep-learning techniques. We conducted experiments that confirm our theoretical findings, demonstrating that even in classical sparse regression settings, the remainder term is too significant be neglected. Furthermore, we applied the proposed method to MRI, achieving significantly better rates on the image support.
Limitations and Future Directions. While our method corrects for the remainder term, larger remainder terms necessitate greater corrections, resulting in wider confidence intervals. The goal is that those intervals should be narrow enough be informative but wide enough to be realistic, given the sample size and variability in the data. Therefore, it is crucial to achieve small remainder term avoid excessively large confidence intervals. Additionally, the accuracy of our method depends on the quality of the estimates for the mean and variance of the remainder term, which improves with more available data. Additionally, the length of the intervals can be minimized over larger parameter set, provided that more data is available. We leave as a future direction to study the sharpness of the proposed confidence intervals and radii for given amount of data. Moreover, we would like to investigate how the length of the confidence intervals could be improved when estimating higher moments. We believe that our method is applicable to a wide variety of deep learning architectures, including vision transformers in MRI, e.g., [77. Testing the generality of the method with oate-oof-hhe-rr architectures for different problems would demonstrate broad usefulness.
10

---

Acknowledgments
We gratefully acknowledge financial support with funds provided by the German Federal Ministry of Education and Research in the grant "SparseMRI3D+: Compressive Sensing und Quantifizierung von Unsicherheiten für die beschleunigte mulippaammetrishhe quantitative Magnerresononzzomorrree (FZK 05M20WOA)".
References
[1] Martin J Wainwright. High-dimensional statistics: A non-asympiotic viewpoint, volume 48. Cambridge university press, 2019.
[2] Christophe Giraud. Introduction to high-dimensional statistics. Chapman and Hall/CRC, 2021.
[3] John Wright and Yi Ma. High-dimensional data analysis with low-dimensional models: Princi- ples, computation,, and applications. Cambridge University Press, 2022.
[4] Scott Chen and David L Donoho. Examples of basis pursuit. In Wavelet Applications in Signal and Image Processing III, volume 2569, pages 564-574. SPIE, 1995.
[5] Robert Tibshirani. Regression shrinkage and selection via the lasso. Journal of the Royal Statistical Society Series B: Statistical Methodology, 58(1):267-288, 1996.
[6] Trevor Hastie, Robert Tibshirani, Jerome H Friedman, and Jerome H Friedman. The elements of statistical learning: data mining. inference, and prediction, volume Springer, 2009.
[7] Keith Knight and Wenjiang Fu. Asymptotics for Lasso-Type Estimators. The Annals of Statistics, 28(5):1356-13777 2000. ISSN 00905364.
[8] C.-H. Zhang and S. S. Zhang. Confidence intervals for low dimensional parameters in high dimensional linear models. Journal of the Royal Statistical Society: Series B (Statistical Methodology), 76(1)2217 2014. ISSN 13697412. doi: 10rss. 12026.
[9] A. Javanmard and A. Montanari. Confidence intervals and hypothesis testing for high- dimensional regression. Journal of Machine Learning Research, 15:2869- 2014.
[10] S. van de Geer, P. Bühlmann, Y. Ritov, and R. Dezeure. On asymptotically optimal confidence regions and tests for high-dimensional models. The Annals of Statistics, 42(3), 2014. ISSN 0090-5364.
doi: 10. 10.1214/14AOSS12.1.
[11] Cun Hui Zhang and Jian Huang. The sparsity and bias of the lasso selection in high-dimensional linear regression. Annals of Statistics, 3664)11567-1554, 2008
[12] Frederik Hoppe, Claudio Mayrink Verdun, Hannah Laus, Felix Krahmer, and Holger Rauhut. Uncertainty quantification for learned ISTA. 2023 IEEE 33rd International Workshop on Machine Learning for Signal Processing (MLSP), pages 1-6. IEEE, 2023.
[13] Pierre C Bellec and Kai Tan. Uncertainty quantification for iterative algorithms in linear models with application to early stopping. arXiv preprint arXiv:2404. 11.4.177855 17856, 2024.
[14] Frederik Hoppe. Felix Krahmer, Claudio Mayrink Verdun, Marion I Menzel, and Holger Rauhut. High--imensional Confidence Regions in Sparse MRL. In ICASSP 2023-2023 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pages 1-5. IEEE, 2023.
[15] A. Javanmard and A. Montanari. Debiasing the lasso: Optimal sample size for Gaussian designs. The Annuls of Statistics, 46(6A), 2018. ISSN 0090-5364. doi: 10.121417-A0S16330
[16] Vishal Monga, Yuelong Li, and Yonina C Eldar. Algorithm unrolling: Interpretable, efficient deep learning for signal and image processing. IEEE Signal Processing Magazine, 38(2): 18-44, 2021.
[17] Jakob Gawlikowski, Cedrique Rovile Njieutcheu Tassi, Mohsin Ali, Jongseok Lee, Matthias Humt, Jianxiang Feng, Anna Kruspe, Rudolph Triebel, Peter Jung, Ribana Roscher, et al. A survey of uncertainty in deep neural networks. Artificial Intelligence Review, 56(Suppl I): 1513-1589, 2023.
[18] Martin Genzel, Ingo Gühring, Jan Macdonald, and Maximilian März. Near-exact recovery for tomographic inverse problems via deep learning. International Conference on Machine Learning, pages 7368-7381. PMLR, 2022.

---

[19] Frederik Hoppe. Felix Krahmer, Claudio Mayrink Verdun, Marion I Menzel, and Holger Rauhut. Uncertainty quantification for sparse Fourier recovery. arXiv preprint 0rXiv:2212.14864 2022.
[20] Ingrid Daubechies, Michel Defrise, and Christine De Mol. An iterative thresholding algorithm for linear inverse problems with sparsity constraint. Communications on Pure and Applied Mathematics: A Journal Issued by the Courant Institute of Mathematical Sciences, 57(11):
1413-1457, 2004.
[21] A. Beck and M. Teboulle. A Fast Iterative Shrinkage-Threshoodding Algorithm for Linear Inverse Problems. SIAM Journal on Imaging Sciences, 2(1): 2009. doi: 10. 1111137777765444 16542..
[22] Karol Gregor and Yann LeCun. Learning fast approximations of sparse coding. In Proceedings of the 27th international conference on machine learning. pages 399-406, 2010.
[23] Simon Foucart and Holger Rauhut. A Mathematical Introduction to Compressive Sens- ing. Springer New York, New York, NY, 2013. ISBN 978-0-8176-4947-00, doi: 10.1007)
778---877-4444--7.
[24] Hui Zou and Trevor Hastie. Regularization and variable selection via the elastic net. Journal of the Royal Statistical Society Series B: Statistical Methodology, 67(2))301-320, 2005.
(25] Ming Yuan and Yi Lin. Model Selection and Estimation in Regression with Grouped Variables. Journal of the Royal Statistical Society Series B: Statistical Methodology, 68(1):49-67, 12 2005.
ISSN 1369-7412. doi: 111111//.167799888 200..0000333
[26] Vladimir Koltchinskii, Karim Lounici, and Alexandre B. Tsybakov. Nuclear-norm penalization and optimal rates for noisy low-rank matrix completion. The Annals of Statistics, 39(5):2302 2329, 2011. doi: 10. 10.1214/11 AA0S8944
[27] Malgorzata Bogdan, Ewout Van Den Berg, Chiara Sabatti, Weijie Su, and Emmanuel Candès. SLOPEEAdaptive Variable Selection via Convex Optimization. The Annals of Applied Statistics, ((65): 2015.
[28] Jianqing Fan and Runze Li. Variable selection via nonconcave penalized likelihood and its oracle properties. Journal of the American statistical Association, 96(456): 1348-1360, 2001.
[29] Cun Hui Zhang. Nearly unbiased variable selection under minimax concave penalty. Annals of Statistics, 38(2):894-942, 2010.
[30] Le Zheng, Arian Maleki, Haolei Weng, Xiaodong Wang, and Teng Long. Does l, Minimizzznnion Outperform & -Miinimizatiin?? IEEE Transactions on Information Theory, 63(1 (111))))66669999 2017.
[31] Alain Rakotomamoniy, Rémi Flamary, Joseph Salmon, and Gilles Gasso. Convergent working set algorithm for lasso with non-convex sparse regularizers. In International Conference on Artificial Intelligence and Statistics, pages 5196-5211. PMLR, 2022.
[32] Alexandre Belloni, Victor Chernozhukov, and Lie Wang. Square-root lasso: pivotal recovery of sparse signals via conic programming. Biometrika, 98(4):911-806, 2011.
[33] Claudio Mayrink Verdun, Oleh Melnyk, Felix Krahmer, and Peter Jung. Fast, blind, and accurate: Tuning-free sparse regression with global linear convergence. In The Thirty Seventh Annual Conference on Learning Theory, pages 3823-3872. PMLR, 2024.
[34] Xudong Li, Defeng Sun, and Kim-Chuan Toh. A highly efficient semismooth Newton augmented Lagrangian method for solving Lasso problems. SIAM Journal on Optimization, 28(1):433 458, 2018.
[35] Christian Kümmerle, Claudio Mayrink Verdun, and Dominik Stöger. Iteratively reweighted least squares for basis pursuit with global linear convergence rate. Advances in Neural Information Processing Systems, 34:2873-2888 2-866, 2021.
[36] Florentina Bunea, Alexandre Tsybakov, and Marten Wegkamp. Sparsity oracle inequalities for the Lasso. Electronic Journal of Statistics, 1:169-194, 2007.
[37] Vladimir Koltchinskii. Sparsity in penalized empirical risk minimization. In Annales de I'IHP Probabilités et statistiques, volume 45, pages 7-57, 2009.
[38] Fei Ye and Cun-Hui Zhang. Rate minimaxity of the Lasso and Dantzig selector for the b, loss in e, balls. The Journal of Machine Learning Research, 11:3519-3540, 2010.
12

---

[39] Garvesh Raskutti, Martin Wainwright, and Bin Yu. Minimax rates of estimation for high- dimensional linear regression over Eg baall. IEEE transactions on information theory. 57(10): 1976-6994, 2011.
[40] Arnak Dalalyan, Mohamed Hcbiri, and Johannes C Lederer. On the prediction performance of the Lasso. Bernoulli, 23(1):552-581, 2017.
[41] Peng Zhao and Bin Yu. On model selection consistency of Lasso. The Journal of Machine Learning Research, 7:2541-2563, 2006.
[42] Martin Wainwright. Sharp thresholds for High-Dimensional and noisy sparsity recovery using l ei-Constrained Quadratic Programming (Lasso). IEEE transactions on information theory, 55 (5):2183-22 2202, 2009.
[43] Simon Foucart, Eitan Tadmor, and Ming Zhong. On the sparsity of LASSO minimizers in sparse data recovery. Constructive Approximation, 57(2):901-919, 2023.
[44] Sai Li. Debiasing the debiased Lasso with bootstrap. Electronic Journal of Statistics, 14: 2298 2037,2337,222 2020.
[45] T Tony Cai and Zijian Guo. Confidence Intervals for High-Dimensional Linear Regression: Minimax Rates and Adaptivity. The Annals of Statistics, pages 615-646, 2017.
[46] Zijian Guo, Domagoj Čevid, and Peter Bühlmann. Doubly debiased lasso: High-dimensional inference under hidden confounding. Annals of statistics, 50(3): 2022.
[47] Pierre C Bellec and Cun-Hui Zhang. De-biasing the lasso with Bernoulli, 28(2):713-743, 2022.
adjustment.
[48] Pierre C Bellec and Cun-Hui Zhang. Debiasing convex regularized estimators and interval estimation in linear models. The Annals of Statistics, 51(2):391 2023.
[49] Frederik Hoppe, Claudio Mayrink Verdun, Hannah Laus, Scbastian Endt, Marion I. Menzel, Felix Krahmer, and Holger Rauhut. Imaging with Confidence: Uncertainty Quantification for High-dimensional Undersampled MR Images. In Proceedings of European Conference on Computer Vision (ECCV), 2024.
[50] Daisuke Ito. Satoshi Takabe, and Tadashi Wadayama. Trainable ISTA for sparse signal recovery. IEEE Transactions on Signal Processing, 67(12):3113 2019.
[51] Kailun Wu, Yiwen Guo, Ziang Li, and Changshui Zhang. Sparse coding with gated learned ISTA. In International conference on learning representations, 2019.
[52] Jialin Liu and Xiaohan Chen. ALISTA: Analytic weights are as good as learned weights in LISTA. In International Conference Learning Representations (ICLR), 2019.
[53] Xiaohan Chen, Jialin Liu, Zhangyang Wang, and Wotao Yin. Hyperparameter tuning is all you need for LISTA. Advances in Neural Information Processing Systems, 34:1 1678-11689, 2021.
[54] Aviad Aberdam, Alona Golts, and Michael Elad. Ada-lista: Learned solvers adaptive to varying models, IEEE Transactions on Pattern Analysis and Machine Intelligence, 44(12):9222-92) 2021.
[55] Ziyang Zheng, Wenrui Dai, Duoduo Xue, Chenglin Li, Junni Zou, and Hongkai Xiong. Hybrid ISTA: Unfolding ISTA with convergence guarantees using free-form deep neural networks. IEEE Transactions on Pattern Analysis and Machine Intelligence, 4523)3322-3244, 2022.
[56] Jonas Adler and Ozan Öktem. Learned primal-dual reconstruction. IEEE transactions on medical imaging, 1322-11332-3332 2018.
[57] Jo Schlemper, Jose Caballero, Joseph V Hajnal, Anthony N Price, and Daniel Rueckert. A deep cascade of convolutional neural networks for dynamic MR image reconstruction. IEEE transactions Medical Imaging, 37(2):491-50 2017.
[58] Hamza Cherkaoui, Jeremias Sulam, and Thomas Moreau. Learning solve TV regularised problems with unrolled algorithms. Advances in Neural Information Processing Systems, 33: 11513-11524, 2020.
[59] Hemant Aggarwal, Merry Mani, and Mathews Jacob. MoDL:: Model-based deep learning architecture for inverse problems. IEEE transactions medical imaging, 38(2):394 40599444444000 2018.
13

---

[60] Risheng Liu, Shichao Cheng. Long Ma, Xin Fan, and Zhongxuan Luo. Deep proximal unrolling: Algorithmic framework, convergence analysis and upplications. IEEE Transactions on Image Processing. 28100):5013-502 2019.
[61] Jian Zhang, Bin Chen, Ruiqin Xiong, and Yongbing Zhang. Physics-inspired compressive sensing: Beyond deep unrolling. IEEE Signal Processing Magazine, 40(1):58-72, 2023.
[62] Simon Arridge, Peter Maass, Ozan Öktem, and Curola-Bibiane Schönlicb. Solving inverse problems using data-driven models. Acta Numerica, 18:1-174, 2019.
[63] Martin Genzel, Jan Macdonald, and Maximilian März. Solving inverse problems with deep neural networks-rrbusttnes included? IEEE transactions on pattern analysis and machine intelligence, 45(1):1119-111 2022.
[64] Emil Y Sidky and Xiaochuan Pan. Report on the AAPM deep-learning sparse-view CT grand challenge. Medical physics, 49(8):4935 49443, 2022.
[65] Ke Li and Jitendra Malik. Learning Optimize. In International Conference on Learning Representations, 2016.
[66] Tianlong Chen, Xiaohan Chen, Wuyang Chen, Howard Heaton, Jialin Liu, Zhangyang Wang, and Wotao Yin. Learning to optimize: A primer and benchmark. Journal of Machine Learning Research, 23(189): 1-59, 2022.
[67] Canberk Ekmekci and Mujdat Cetin. Uncertainty quantification for deep unrolling-basee computational imaging. IEEE Transactions on Computational Imaging,
2022.
[68] Yarin Gal and Zoubin Ghahramani. Dropout as a bayesian approximation: Representing model uncertainty in deep learning. In international conference on machine learning, pages 1050-1059, PMLR, 2016.
[69] Glenn Shafer and Vladimir Vovk. A tutorial on conformal prediction. Journal of Machine Learning Research, 9(3), 2008.
[70] Vladimir Vovk, Alexander Gammerman, and Glenn Shafer. Algorithmic learning in a random world, volume 29. Springer, 2005.
[71] Gilad Kutiel, Regev Cohen, Michael Flad, Daniel Freedman, and Ehud Rivlin. Conformal prediction masks: Visualizing uncertainty in medical imaging. In International Workshop on Trustworthy Machine Learning for Healthcare, pages 163-176. Springer, 2023.
[72] Anastasios N Angelopoulos, Amit Pal Kohli, Stephen Bates, Michael Jordan, Jitendra Malik, Thayer Alshaabi, Srigokul Upadhyayula, and Yaniv Romano. Image-to-image regression with distribution-free uncertainty quantification and applications in imaging. In International Conference on Machine Learning, pages 717-730, PMLR, 2022.
[73] Olaf Ronneberger, Philipp Fischer, and Thomas Brox. U-net: Convolutional networks for biomedical image segmentation. In Medical image computing and computer-assisted intervention-MIICCA. 2015: 18th international conference, Munich, Germany, October 5-9, 2015, proceedings. part III 18, pages 234-241 Springer, 2015.
[74] Jure Zbontar, Florian Knoll, Anuroop Sriram, Tullie Murrell, Zhengnan Huang, Matthew J. Muckley, Aaron Defazio, Ruben Sterm, Patricia Johnson, Mary Bruno, Marc Parente, Krzysztof J. Geras, Joe Katsnelson, Hersh Chandarana, Zizhao Zhang, Michal Drozdzal, Adriana Romero, Michael Rabbat, Pascal Vincent, Nafissa Yakubova. James Pinkerton, Duo Wang, Erich Owens, C. Lawrence Zimick, Michael P. Recht, Daniel K. Sodickson, and Yvnnn Lui. fastMRI: An Open Dataset and Benchmarks for Accelerated MRI, 2019.
[75] Florian Knoll, Jure Zbontar, Anuroop Sriram, Matthew J. Muckley, Mary Bruno, Aaron Defazio, Mare Parente, Krzysztof J. Geras, Joe Katsnelson, Hersh Chandarana, Zizhao Zhang, Michal Drozdzaly, Adriana Romero, Michael Rabbat, Pascal Vincent, James Pinkerton, Duo Wang, Nafissa Yakubova, Erich Owens, Lawrence Zitnick, Michael Recht, Daniel K. Sodickson, and Yvonne W. Lui. fastMRI: A Publicly Available Raw k-Space and DICOM Dataset of Knee Images for Accelerated MR Image Reconstruction Using Machine Learning. Radiology: Artificial Intelligence, 2(1): 190007, 2020. doi: 10. wtw.rryyaiii201900eee
PMID: 32076662.
[76] Zhou Wang, Eero P Simoncelli, and Alan C Bovik. Multiscale structural similarity for im- age quality assessment. The Thirty-Sevetth Asilomar Conference on Signals, Systems & Computers, 2003, volume 2, pages 1398-1402 Icce, 2003.
14

---

[77] Kang Lin and Reinhard Heckel. Vision transformers enable fast and robust accelerated mri. In International Conference on Medical Imaging with Deep Learning, pages 774-795. PMLR, 2022.
[78] John G. Saw, Mark C. K. Yang, and Tse Chin Mo. Chebyshev Inequality with Estimated Mean and Variance. The American Statistician, 38(2):130-332, 1984. ISSN 00031305.
[79] Santiago Aja-Fernándzz and Gonzalo SanassSnnhhzz--een MRI. Switzerland: Springer International Publishing, 2016.
Statistical analysis of noise in
[80] Tingni Sun and Cun-Hui Zhang. Scaled sparse linear regression. Biometrika, 99(4):879-898, 2012.
[81] Stephen Reid, Robert Tibshirani, and Jerome Friedman. A study of error variance estimation in lasso regression. Stafist. Sinica, pages 35-67, 2016.
[82] Christopher Kennedy and Rachel Ward. Greedy variance estimation for the LASSO. Appl. Math. Optim., 121111161 2020.
[83] Christophe Giraud, Sylvie Huet, and Nicolas Verzelen. High-dimensional regression with unknown variance. Statist. Sci., 27(4):500-518; 2012.
[84] Lee Dicker. Variance estimation in high-dimensional linear models. Biometrika, 101(2): 269-284, 2014.
[85] X Liu, S Zheng, and X Feng. Estimation of error variance via ridge regression. Biometrika, 107 (2):41-4883 2020.
[86] Guo Yu and Jacob Bien. Estimating the error variance high-dimensional linear model. Biometrika, 106(3)5533-546, 2019.
[87] Bartolomeo Stellato, Bart P. G. van Parys, and Paul J. Goulart. Multivariate Chebyshev Inequality With Estimated Mean and Variance. The American Statistician, 71(2):123-127,2000 2017. 186559.
ISSN 0003- 1305. doi: 10. (080/00031305.2011
[88] Esa Ollila, David E. Tyler, Visa Koivunen, and H. Vincent Poor. Complex Elliptically Symmetric Distributions: Survey, New Results and Applications. IEEE Transactions Signal Processing, 60(11):55775565, 2012. doi: 10.1 109/TSP.201..2212453
[89] R. Becker, E. Candès, and C. Grant. Templates for convex cone problems with applications to sparse signal recovery. Mathematical programming computation, 363):15--11 2011.
15

---

Supplementary material to the paper Non-Asymptotic Uncertainty Quantification in High- Dimensional Learning.
In this supplement to the paper, we present in Section A detailed discussion about aspects of the main result that not mentioned in the main body of the paper. Moreover, Section B presents the proof Theorem 2 and Theorem 3 The former establishes data-driven confidence intervals, while the
latter assumes the remainder component to be approximated by Gaussian distribution. Appendix
C constructs confidence intervals similar to Theorem g under the assumption of a heavy-tailed distribution. Section [ we confirm our theoretical findings with several numerical experiments for classical high-ding dighedimenii regression as well as model-based neural networks. In Section E] we visualize the approximate Gaussian distribution of the remainder terms, demonstrating the applicability of Theorem 3 in relevant settings.
A Further Discussion of Main Result
In this appendix, we provide an in-depth discussion of various technical aspects and practical considerations regarding our main theoretical results, Theorem [ including analysis of confidence interval radii, probabilistic interpretations, and aaplemmentttion-oreenee relaxations of the theoretical assumptions.
Length of radius: To minimize the length of the radius, 7 E (0, minimizer of the problem
should be chosen the
(MEM
12 12(1 75)a (ōR)s
min (01-0
log
mm
In order to minimize over a large set for a given significance level a, a large number of data is needed. For fixed estimates S, and (d%)). more data leads to a potentially smaller confidence interval length. If we assume Rý it follows that 3, 0 and (o%) O. Then, 7 is valid choice, is well-defined and is minimized. In this case, the
for which the function <(MEMM') log 7y Vm
radius coincides with the asymptotic radius derived in 9 10 15| (except for that these works handle the real case) and the ones in [12] with M = IN In this sense, the asymptotic confidence intervals can be seen as a special case of the proposed method.
The significance level depends 7j and / to assure at :) to be well-defined. For large dataset
a(1)) (1), i.e. if is large, then it holds that lim (a(a) = lim
+ (1-3))--1
14000
1-000
ViI-g)a
Probabilistic discussion: The probability in (5) is over the randomness of the noise as well as Q The confidence circles (a()) consist of two random variables, the debiased estimator &" and the radius ry (a). The former depends on the random noise and potentially training data, while the latter depends the estimators S; and ân, which in turn depend on both the noise and the data x(1)) x()),......!), (f),
crucial requirement of applying the empirical version of Chebyshev's 's inequality (78 is the independence and identical distribution of the variables I2, |n("). Therefore, it is essential that the estimator function X is independent of the data (1) ..... (t)) To achieve this, we train the estimator function X using a dataset that is independent of the data (1)) ..... ....,(()) used for estimating R(!,,..., R(), However, the mean and variance of [ and hence of [R(( depend on the variance of the noise e(1), ie. o². Thus, different noise levels a require a new estimation the mean and variance of |. Throughout this paper, we assume the noise level to be fixed and known. The latter assumption is motivated by two factors. First, the size of the confidence intervals relies Given that the primary focus this paper is to determine the size of the confidence intervals based on the remainder term R(1), we seek to mitigate other influencing factors such as noise level estimation. Second, in domains like medical imaging, there is substantial knowledge about the noise level. For instance, the noise level in MRI can directly measured from the scanner [79]. If the noise level
unknown, there are methods to estimate it. In the debiased LASSO literature, the most used method
16

---

is the scaled LASSO [8)] Other methods for sparse regression, either in the LASSO context or more general for high-dia nigh-dimensional models, are [6].
Relaxation of assumptions in practice: practice, is often the case, that | |..., R are identical distributed resulting in [i] =... = μ.N and (a%) = = )o Although the proof requires independence of the | there are cases when it might suffice to relax this assumption by estimating the mean and variance pixel-wise uniformly, i.e.,
N and o I. ΣE
1 ΣE $
$ s= S=I.N I.N j=1
R
S)
(8)
In addition saving computational resources, accuracy improves due the higher number of samples. Furthermore, instead of solving the optimization problem ( for every j E (I,...,NK it might be good idea choose 7/1 = YN as the minimizer of
(MEM
1 N j=1
log
Tjaa + ai ((1 (1-7))a)
(nn)
(9)
VmN
7
Then, one 7 can be used for computing the potentially different radii ry()).
Type of uncertainty: The decomposition of the estimation error into a Gaussian term and a remainder term allows for handling both types of uncertainty, aleatoric and epistemic, almost separately. With the Gaussian term W. we quantify the aleatoric uncertainty from the inherent measurement noise. The remainder term R handles the epistemic uncertainty, which we quantify using a purely data-driven approach since for two different backward models (e.g., two different neural networks), it can be used to compare the estimation error of both models with respect to the ground truth. In this sense, our technique is rather a more general inferential uncertainty method.
B Proofs
(++1 Cy(a) is equivalent to [(3))(+1] 2/4111 s rr)))). Proof of Theorem 2 The statement x.
To prove we show that
P (1+1) -z (+1) 2 ry((a) sa
-x
In the next step, write the radius r(a) the sum r(a) = n (a) r (a). According to the decomposition (62) -z (1+1) W + Rg we obtain for fixed j [l,...,,]] N
P (162)) (+1)) 23 (+11) 2 , (a) +r (a)) = P (IW, + Rg 2 r (a) + +r)(aa))
W <P (IW,) |Ryl 2 r (a) + +r)((a)) (a)) = IP ( 2 (a)) + (12)(2r((()))
where the last step follows from the pigeonhole principle. To estimate the first summand, we set (a) "MMSM')) Vm
Since [W ~ Rice 0. a(MSMM )
log 7ya/
we obtain
22m
2m
x2m
P (IW 2 r (2 (a:)
T exx
dr=
exp(-u)du
02 Σ
(M
(a)
(r (m
(r. (a))
+2(MEM*) (- a.a.
exp
For estimating the term PP (|R11 2 (a)), we set (() = c((a) (ōn); + S... This choice leads to
P =
[Ryl - S, 2 73 -8,) < P (|1y)1 - 5, ž 2イ - S,)
| - 53 (a(aa) - 3
=P ( ||]] - S
zala)
=
(ōR)s
(ŌR);
(ōR);
17

---

Now, we apply an empirical version of Chebyshev's inequality 778 87
This leads to
Ríí 1 S
1)(2 - 1 la)())2)
a((a)
min
1 : 1
(ān)s
12a(a))
12- - 1 la((a)2
12 + 1[1 (11-3)/a-
1
min
1,
min
1,
(2)2
1(12 (12-1) 1)
1(1 )a-
+ (117,)a-11
min
min (1-11))a =(1 (1-3g)0
K1-7,
where we used in the last step, that (i- < aa < 1. To summarize,
P (1 xz 1 1 1z(aa) S ((W (a)) + 2 rf(a))
3 (1-7,)a
Proof of Theorem 3 Since W CN(O, d EMMM* and CN 1 ER) the estimation error H x W + follows again a multivariate normal distribution with zero mean and covariance L + ER). By exploiting the Gaussian distribution, we obtain
m
matrix
m
2m
x²m
P (IW, + Ry] 25 (a)) )2(MMM )i + (ER)ss
I exp
(MLMM (ER)s
dr
(()))
(a)m +2(MMM*) + (ER)s)
exp
2 (a))
2MMMM by3 (Bndg)
Thus, have
(((a)3m (2(MM)) + (ER)s)
P(1y 21]1 > (r)((aa) < exp
which needs be equal
0. Therefore,
(() (o(MÉM, + (ER);))
log
mm
C Heavy-Tailed Version of Theorem 3
The following version of Theorem gives an example of how to derive the radii of the confidence intervals in the case of a known, -tiled distribution of the remainder term.
Theorem 4. Let 8" E CN be debiased estimator for E CN with a remainder term following complex d-istributionn with degree of freedom 2, i.e. (o, En). Then, C,(a) = {2 C 1 - 221 S r,(a)} with radius
ry(a)= (MMEM) 1/2 3j
ERRiJU
log 7
+
(1-7)))-211 -2/va-2 1
Vm
2
valid, i.e. PP (z, Cy(a)) 2 1-a. a.
18

---

Proof.
We can bound the estimation error jey zzj analogously to the proof of Theorem 2 by P( - 2y 2 ry(a)) P(W,, + R,} ry S P((W)) > r (a)) + P([R)) (a))
The distribution of |( > r (a)) is determined by the Gaussian noise and can be computed as
ry (a) (MEM))) 1/2 1j
1
log
vm
Yja
similarly to Theorem 2. If R (0, ER), then the marginal distribution R2 is also complex Moreover, the probability density function of [Ry] is
t-distributed, i.c. R2 ~ (0, (ER)ss) (see (8)
2r
2r
(v/2+1)
[(r) (En)ss
1
v ((En)s
Hence,
P([R)) > of(()))) = / f(r)dr=
(ER)sg + +1) 1/2
+1) 1/2
2r
2r2
(5(()
ra(a)
(En)sy
Setting this probability equal to (1-1) (1-7g)a requires
(f()) = (ER)iju
(Emnii v(1 (11- 1 -2/10
Since we assumed > 2 the inequality (1 75)0-/121 22 >
holds. Combining rj(a)
ry (a) + (f((a) concludes the proof.
D Further Numerical Evaluation
To confirm our theoretical findings claiming that the incorporation of the bias component renders the confidence intervals more robust, we present additional numerical experiments here.
UQ for Classical Model-Based Regression For the experiments described here, we use Tfocs Analogous to the experiment described in Section [.1] we run further experiments in the classical sparse regression setting when the measurement matrix is a Gaussian and subsampled Fourier matrix. The different settings including the results can be found in Table The results show that the Gaussian adjustment of our proposed method significantly increases the hit rates, especially on the support, while moderately inereasing the confidence interval length. Our data-driven adjustment achieves even better hit rates, but the confidence intervals are larger. Although in well-posed settings, like the second column of Table 1 the hit rates h (0.05) based asymptotic confidence intervals lead overall to almost 95%, however on the support, which are the crucial features, the asymptotic hit rates fail. In particular, our corrections column. The hit rates for the asymptotic CIs and the corrected ones with Gaussian adjustment are visualized in more detail in Figure 4
essential in ill-posed regression problems as the third Gaussian
UQ for MRI Reconstruction with Neural Networks this section, we present more experiments for UQ for MRI reconstruction with neural networks, Our experimental settings, well as code for this experiment, are based on the paper and by [18]. The dataset used for conducting the
experiments is the fastMRI single-coil knee dataset. For documentation, see [74] 75).
Table 2 represents the results obtained by learning the reconstruction function X using the It-net with 8 layers, with 60%, 40% and 30% radial undersampling and for noise levels obtained by adding complex gaussian noise with standard deviation
60 and a 84, respectively. Similarly, Table showws the results obtained by the U-Net. In Figure G the asymptotic hit rates and the Gaussian adjusted ones for the 95% confidence level are compared in a box plot for each experiment. Note that for these experiments, we construct only the confidence intervals for the real part of the image, as the
19

---

530
64
Oran CaLnEr ioumant
Dap We
A0P
Crarh DOLTTM Hqureer
(a)
(b)
(c)
K55
60
(d)
(c)
4.5
60
660
024
Crain Coutie dammen
(h)
2.15
2110
(k)
Figure 4: Box plots for hit rates of sparse regression experiments. The settings those described in Table The first row presents the hit rates over all components and the second the hit rates of the support, e.g.. and and correspond to the first column the table, 44b and 4h the second one and so forth. In each plot the left box represents the asymptotic hit rates, the right one the Gaussian adjusted hit rates. The horizontal line marks the desired 95%.
fastMRI dataset provides only real-valued data. Due to this artifact, the investigation of the imaginary part's distribution is not possible, and the Gaussian-adiin adjjssted confidence intervals are not applicable to the magnitude. For this reason, and ensure valid comparison of the three confidence interval types, we decided to investigate the confidence intervals for the real part, even in the asymptotic and data-driven setting.
It-Net and U-Nets are trained with a combination of the MS-SSIM-loss 76,, the flsss and the Adam optimizer with learning rate of 5e epsilon of 1e-1 and weight decay parameter le
20

---

type
Gaussian
Fourier
feature dimension
1000
10000
1000
10000
100000
undersampling
50%
40%
60%
40%
60%
50%
sparsity
7.5%
2%
10%
5%
10%
5%
relative noise
15%
10%
20%
15%
5%
10%
R/W æ2-ratio
0.7062
0,5117
0.9993
0.5446
0.5924
0.4701
R/W & -ratio
0.8191
0.5161
1.1581
0.5752
0.6088
0.4794
average asympt. radius " (0.05)
0.0116
0.0027
0.0049
0.0130
0.0011
0.0008
av. radius r (0.05) (Thm
0.0142
0.0031
0.0069
0.0148
0.0013
0.0009
av. radius r(0.5) (Thm
0.0304
0.0060
0.0155
0.0295
0.0026
0.0016
(0.05)
0.9437
0.9353
0.8692
0.9582
0.9246
0.9444
(0.05)
0.9787
0.9684
0.9691
0.9799
0.9665
0.9687
h(0.05)
1
0,9999
h (0.05)
0.7661
0.8941
0.6783
0.8629
0.8745
0.9041
(0.05)
0.8852
0.9421
0.8948
0.9226
0.9396
0.9425
hg(0..0)
0.9999
0.9999
0.9998
Table Experiments for sparse regression for Gaussian and Fourier matrix. Every experiment uses 500 estimation and 250 evaluation data.
(c)
Figure 5: Knee MRI groundtruth image from the fastMRI dataset 5a 74 5b and undersampled k-space data 5c
75
radial sampling mask
The It-Nets were trained for 15 epochs, and the U-Nets were trained for 20 epochs, both with batch size 40. Every U-Net has 2 input and output channels, 24 base channels, and encodes the image to size of 20 x 20 and at most 384 channels. The It-Net It-Net employs the U-Net in each layer as residual network and has data consistency part around each U-Net in every layer.
Comparing the tables, the It-Net has, in general, better hit rates as well as a better R/W ratio than the U-Net due to its more accurate reconstruction. Further, the hit rates for all the pixels are higher than those obtained only for the support. For achieving reliable results for safety-critical applications, obtaining hit rates higher than the confidence level is crucial, especially the support. Otherwise, one might achieve certain confidence level overall but cannot trust the pixels of interest.
The experiments were conducted using Pytorch 1.9 on desktop with AMD EPYC 7F52 16-Cor CPUs and NVIDIA A100 PCIc030 030 40GB GPUs. The code for the experiments can be found in the supplementary material. The execution time of the code is around 5 hours for each It-Net, 2 hours for each U-Net, and around 30 minutes for the rest of each experiment. So, in total, this gives us time of 48 hours for the MRI reconstruction experiments. The execution time for the classical model-based regression experiments takes to 30 minutes each; therefore, in total it is less than hours.
E Distribution Visualization of Remainder Term
In Figure 7 we present series of histograms illustrating the empirical distribution of the remainder term's real part across all experimental settings in sparse regression, conducted in this
21

---

8
0.80
0403
(b)
(c)
0290
0.00
0200
440
(d)
(c)
1.00
1.50
1.00
0.0
3.0
0.0
3.30
3.80
0.00
(h)
1.00
1.0
4.9
6000
(k)
Figure 6: Box plots for hit rates of neural network experiments. The settings are those described in Table 2 and Table The first row presents the hit rates for the different It-nets for 95% confident intervals and the second the hit rates for the U-Nets for 95% confident intervals, e.g., 6a and 6g correspond to the first columns of the tables. In each plot the left box represents the asymptotic hit rates, the right the Gaussian adjusted ones.
paper. These histograms provide evidence that the remainder term can be approximated by a Gaussian distribution, with the approximation becoming increasingly precise the dimensionality increases. Across low-dimensional scenarios, the empirical distributions exhibit some deviations from the Gaussian form, but these discrepancies diminish the dimensionality grows larger. In high-dimensional regimes, the empirical distributions demonstrate an exceptional degree of convergence to the Gaussian approximation. This close alignment lends strong support to the va- lidity of the key assumption of Theorem 3 allowing a Gaussian adjustment to the confidence intervals.
In Figure we present series histograms representing the empirical distribution of the remainder term's 's real part for the six different experimental settings, for the U-Net, conducted in this paper.
22

---

undersampling
60%
40%
30%
noise level
15%
10%
12%
8%
10%
7%
R/W E2-ratio
0.3558
0.3800
0.6332
0.6922
0.8759
0.5759
R/W -rato
0.3842
0.4929
0.6268
0.6924
1.0614
0.6282
h(0.05)
1.0
1.0
1.0
1.0
1.0
1.0
(0.05)
0.9797
0.9791
0.9826
0.9818
0.9831
0.9770
hw (0.05)
0.9726
0.9689
0.9714
0.9649
0.9690
0.9518
h(0.1)
0.9998
0.9998
0.9999
0.9999
1.0
0.9999
(0.1).)
0.9486
0.9485
0.9544
0.9539
0.9553
0.9448
hw
0.9359
0.9306
0.9338
0.9241
0.9299
0.9033
(0.0005)
1.0
0.9999
1.0
1.0
1.0
1.0
(0.00)5)
0.9739
0.9711
0.9789
0.9729
0.9798
0.9707
hg (0.05)
0.9656
0.9589
0.9663
0.9520
0.9642
0.9421
(0(1).)
0.9997
0,9995
0.9999
0.9998
0.9999
0.9998
(0.1)
0.9385
0.9353
0.9476
0.9392
0.9493
0.9347
hg (0.1)
0.9246
0.9152
0,9257
0,9053
0.9225
0.8906
Table 2: Experiments for It-Net with 8 iterations. Results of hit rates averaged over & 100 samples.
undersampling
60%
40%
30%
noise level
15%
10%
12%
8%
10%
7%
R/W ratioo
0.2747
0.3976
0.6182
0.6671
1,2641
1.3399
R/W Ex -rato
0.3923
0.4292
0.5681
0.7082
1.1668
1.2501
h(0.05)
0.9999
1.0
1.0
1.0
1.0
1.0
(0.05)
0.9657
0.9688
0.9737
0.9767
0.9768
0.978
hw (0.05)
0.9569
0.9552
0.9583
0.9535
0.9517
0.9418
h(0.1)
0.9993
0.9997
0.9998
0.9999
1.0
1.0
(0.1)
0.9238
0.9296
0.9377
0.9443
0.9432
0.9465
(0.1)
0.9098
0.9077
0.9125
0.9060
0.9022
0.8886
hs(0.05)
0.9999
0.9999
1.0
1.0
1.0
1.0
(0.05)
0.9679
0.9677
0.975
0.9727
0.9779
0.9757
h (0.05)
0.9594
0.9541
0.9602
0.9483
0.9542
0.9398
(0.1.
0.9994
0.9996
0.9999
0.9998
1.0
0.9999
(0.1)
0.9276
0.9282
0.9401
0.9388
0.946
0.9446
h3 (0.1)
0.9141
0.9064
0.9156
0.9002
0.9061
0.8876
Table 3: Experiments for U-Nets. Results of hit rates averaged over
100 samples.
Figure 9 represents the histograms for the It-Net experiments. In most scenarios, the real part of the remainder term is Gaussian distributed with mean The only exceptions are Figures and which correspond to the U-Net experiments with 30% undersampling.
23

---

(a)
(b)
(c)
(d)
(f)
Figure 7: Histograms showing the empirical distribution of the remainder component. The real part of plotted as histogram and the red line corresponds Gaussian fit. One realization of the remainder term is visualized for each of the experiments described in Table (7 corresponds to the first column, 7b to the second column and on. The plots for the imaginary part look similar.)
(a)
(b)
(c)
25
60
-77 +000
05
(d)
(c)
(f)
Figure 8: Histograms showing the empirical distribution of the remainder component. The real part of R is plotted as histogram and the red line corresponds to Gaussian fit. One realization of the remainder term is visualized for each of the experiments described in Table corresponds the first column, to the second column and so on.)

---

26
(a)
(b)
(c)
05
(d)
(c)
(f)
Figure 9: Histograms showing the empirical distribution of the remainder component. The real part of R is plotted as histogram and the red line corresponds to Gaussian fir. One realization of the
remainder term is visualized for each of the experiments described in Table 2 first column, 9h to the second column and so on.
Da corresponds the
25

---

NeurIPS Paper Checklist
1. Claims
Question: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?
Answer: [Yes]
Justification: In this paper will develop theory for non-asymptotic uncertainty quantification in high-dimensional learning and, as described the abstract, we also apply such theory to neural networks and learning algorithms, bridging theory and practice.
Guidelines:
The answer NA means that the abstract and introduction do not include the claims made in the paper.
The abstract and/or introduction should clearly state the claims made, including the contributions made in the paper and important assumptions and limitations. A No or
NA answer to this question will not perceived well by the reviewers.
The claims made should match theoretical and experimental results, and reflect how much the results
be expected to generalize other settings.
It is fine to include aspirational goals as motivation long it is clear that these goals are not attained by the paper.
2. Limitations
Question:
Does the paper discuss the limitations of the work performed by the authors?
Answer: [Yes]
Justification: The limitations of this paper are discussed in Section g In particular, we discussed that larger remainder terms necessitate greater corrections, resulting wider confidence intervals. Also, the accuracy of our method depends on the quality of the estimates for the mean and variance of the remainder term, which improves with more available data. Lastly, we left the sharpness of the provided results future work.
Guidelines:
The answer NA means that the paper has no limitation while the answer No means that the paper has limitations, but those are not discussed in the paper.
The authors are encouraged to create separate "Limitations" section in their paper.
The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g., independence assumptions, noiseless settings, model vell-specifiiattio, asymptotic approximations only holding locally). The authors should reflect on how these assumptions might be violated in practice and what the implications would be.
The authors should reflect on the scope of the claims made, e.g., if the approach was only tested few datasets with few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.
The authors should reflect on the factors that influence the performance of the approach. For example, facial recognition algorithm may perform poorly when image resolution is low or images are taken in low lighting. Or speech-to-text system might not be used reliably to provide closed captions for online lectures because it fails handle technical jargon.
The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size.
If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and faimess.
While the authors might fear that complete honesty about limitations might be used by reviewers as grounds for rejection, worse outcome might be that reviewers discover limitations that aren't acknowledged the paper. The authors should use their best judgment and recognize that individual actions in favor of transparency play an impor- tant role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed to not penalize honesty concerning limitations.
26

---

3. Theory Assumptions and Proofs
Question: For each theoretical result, does the paper provide the full set of assumptions and a complete (and correct) proof?
Answer: [Yes]
Justification: All theoretical results contain proofs, in detail Theorem 2 and Theorem are proven in Section and Theorem iss an informal version of Theorem 2 All assumptions are made in the statements of the theorems, and all results the proofs rely are referenced.
Guidelines:
The answer NA means that the paper does not include theoretical results.
All the theorems, formulas, and proofs in the paper should be numbered and cross- referenced.
All assumptions should be clearly stated or referenced in the statement of any theorems.
The proofs can either appear in the main paper or the supplemental material, but if they appear in the supplemental material, the authors are encouraged to provide short proof sketch provide intuition.
Inversely, any informal proof provided in the core of the paper should complemented by formal proofs provided in appendix or supplemental material.
Theorems and Lemmas that the proof relies upon should be properly referenced.
4. Experimental Result Reproducibility
Does the paper fully disclose all the information needed to reproduce the main ex- perimental results the paper to the extent that affects the main claims and/or conclusions of the paper (regardless of whether the code and data are provided not)?
Question:
Answer: [Yes]
The main information about the experimental results and how they were obtained can be found in Section [S] further information can be found in the Appendix in Section D We will also provide github containing the code.
Justification:
Guidelines:
The answer NA means that the paper does not include experiments.
If the paper includes experiments, a No answer to this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.
If the contribution is dataset and/or model, the authors should describe the steps taken to make their results reproducible or verifiable.
Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution a novel architecture, describing the architecture fully might suffice, or if the contribution is specific model and empirical evaluation, it may be necessary to either make it possible for others to replicate the model with the same dataset, provide access to the model. In general. releasing code and data is often one good way to accomplish this, but reproducibility can also be provided via detailed instructions for how replicate the results, access hosted model (e.g., in the of a large language model), releasing of a model checkpoint, or other means that are appropriate to the research performed.
While NeurIPS does not require releasing code, the conference does require submis- sions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example
(a) the contribution is primarily a new algorithm, the paper should make clear how reproduce that algorithm. (b) If the contribution is primarily the architecture clearly and fully.
model architecture, the paper should describe
(c) the contribution a new model (e.g., a large language model), then there should either be way to access this model for reproducing the results or way to reproduce the model (e.g., with an open-source dataset instructions for how to construct the dataset).
27

---

(d)
We recognize that reproducibility may be tricky in some cases, in which case authors are welcome describe the particular way they provide for reproducibility. In the case of closed-source models, it may be that access to the model is limited in some way (e.g., to registered users), but it should be possible for other researchers to have some path to reproducing or verifying the results.
5. Open access to data and code
Question: Does the paper provide open access to the data and code, with sufficient instruc- tions to faithfully reproduce the main experimental results, as described in supplemental material?
Answer: [Yes]
Justification: We will release Github containing all code and explanation how reproduce our results once the paper is published, for now we attached the code as supplemetary material.
Guidelines:
The answer NA means that paper does not include experiments requiring code.
Please the NeurlPS code and data submission guidelines (https://nips.cr
Seallic/giies/Codee http///Suuusisssssooooo.oo..
for more details.
While we encourage the release of code and data, we understand that this might not be possible, so "No" an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (c.g., for new open-source benchmark).
The instructions should contain the exact command and environment needed to run to reproduce the results. See the NeurIPS code and data submission guidelines (https
ptapps..c/publii/guuidesss Sea.eeeeee/oooeSeeeeeeeeeeeeeeee wwtssSuummisiiiiniio.. sww.iiiiiooiocooooooooooo for more details.
The authors should provide instructions on data access and preparation, including how access the raw data, preprocessed data, intermediate data, and generated data,
The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. If only subset of experiments are reproducible, they should state which ones are omitted from the script and why.
At submission time, to preserve anonymity, the authors should release anonymized versions (if applicable).
Providing as much information as possible in supplemental material (appended the paper) is recommended, but including URLs to data and code is permitted.
6. Experimental Setting/eetails
Question: Does the paper specify all the training and test details (c.g., data splits, hyper- parameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?
Answer: [Yes]
Justification: The main experimental settings be found in Section 5 more details about the experimental settings can be found in Appendix D and in the code.
Guidelines:
The answer NA means that the paper does include experiments.
The experimental setting should be presented in the core the paper to a level detail that is necessary to appreciate the results and make sense of them.
The full details can be provided either with the code, in appendix, as supplemental material.
7. Experiment Statistical Significance
Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?
Question:
Answer: [Yes]
Justification: Yes, the corresponding boxplots and confidence intervals can found in Figures 2 6
28

---

Guidelines:
The answer NA means that the paper does not include experiments.
The authors should answer "Yes" if the results are accompanied by error bars, confi- dence intervals, or statistical significance tests, least for the experiments that support the main claims of the paper.
The factors of variability that the error bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall run with given experimental conditions).
The method for calculating the error bars should be explained (closed form formula, call to a library function, bootstrap, etc.)
The assumptions made should be given (e.g.. Normally distributed errors).
It should be clear whether the error bar is the standard deviation or the standard error of the mean.
is OK to report 1-sigma error bars, but one should state it. The authors should preferably report a 2-sigma error bar than state that they have 96% CL, if the hypothesis of Normality of errors is not verified.
For asymmetric distributions, the authors should be careful not to show in tables or figures symmetric error bars that would yield results that out of range (e.g. negative error rates).
If error bars are reported in tables or plots, The authors should explain in the text how they were calculated and reference the corresponding figures or tables in the text.
8. Experiments Compute Resources
For each experiment, does the paper provide sufficient information on the com- puter resources (type of compute workers, memory, time of execution) needed reproduce the experiments?
Question:
Answer: [Yes]
Justification: All this information provided in the Appendix D Further, the paper did not require more computation and experiments than what is presented in the paper.
Guidelines:
The answer NA means that the paper does not include experiments.
The paper should indicate the type of compute workers CPU or GPU, internal cluster, or cloud provider, including relevant memory and storage.
The paper should provide the amount of compute required for each of the individual experimental rums as well as estimate the total compute.
The paper should disclose whether the full research project required more compute than the experiments reported didn't 'tttt make it into the paper).
paper (e.g., preliminary or failed experiments that NeurlPS Code of Ethics httpp::/neuurips.c http://wwww.pp...pp//ccccccsaaaa
9. Code Of Ethics
Question: Does the research conducted the paper conform, in every respect, with the
Answer: [Yes]
Justification: Yes, we checked the code ethics and our paper conforms with every point in the code, also made sure to preserve anonymity.
Guidelines:
The answer NA means that the authors have not reviewed the NeurlPS Code of Ethics.
If the authors answer No, they should explain the special circumstances that require deviation from the Code of Ethics.
The authors should make sure to preserve anonymity (e.g., there special consid- eration due to laws or regulations in their jurisdiction).
10. Broader Impacts
Question: Does the paper discuss both potential positive societal impacts and negative socictal impacts of the work performed?
29

---

Answer: [Yes]
Justification: This paper is developing a rigorous framework for confidence intervals for dishesiimenio inverse problems. We believe that this will improve the diagnostics with medical images, as discussed Section 6 We do not believe that such techniques could have negative societal impact.
Guidelines:
The answer NA means that there no societal impact of the work performed.
If the authors answer NA or No, they should explain why their work has no societal impact why the paper does not address societal impact.
Examples of negative societal impacts include potential malicious unintended uses (e.g.. disinformation, generating fake profiles, surveillance), fairness considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.
The conference expects that many papers will be foundational research and not tied particular applications, let alone deployments. However, if there is direct path to any negative applications, the authors should point it out. For example, it is legitimate point out that an improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people train models that generate Deepfakes faster.
The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology is being used as intended but gives incorrect results, and harms following from (intentional or unintentional) misuse of the technology.
there are negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition attacks, mechanisms for monitoring misuse, mechanisms to monitor how a system learns from feedback over time, improving the efficiency and accessibility of ML).
11. Safeguards
Does the paper describe safeguards that have been put in place for responsible release of data models that have high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?
Question:
Answer: [NA]
Justification: Since the paper develops confidence intervals for machine learning problems with solid theoretical foundations, there is no risk misuse and no safeguards are necessary.
Guidelines:
The answer NA means that the paper poses no such risks.
Released models that have high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere to usage guidelines or restrictions to access the model or implementing safety filters.
Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.
We recognize that providing effective safeguards is challenging, and many papers do not require this, but we encourage authors to take this into account and make best faith effort.
12. Licenses for existing assets
Are the creators or original owners of assets (c.g., code, data, models), used in the paper, properly credited and are the license and terms of use explicitly mentioned and properly respected?
Question:
Answer: [Yes]
which we cited according their homepage. We used publicly available code belonging to the papers 63 and (89 which we
Justification: We used the NYU fastMRI dataset 74 75
also cited at the according places (Section 5 or Section D)
30

---

Guidelines:
The answer NA means that the paper does not existing assets.
The authors should cite the original paper that produced the code package dataset.
The authors should state which version of the asset is used and, if possible, include URL.
The name of the license ge.g., CC-BY 4.0) should be included for each asset.
For scraped data from a particular source (e.g., website), the copyright and terms of service of that source should be provided.
If assets are released, the license, copyright information, and terms of use in the package should be provided. For popular datasets, paperswi thcode. com datasets has curated licenses for some datasets. Their licensing guide can help determine the license dataset.
For existing datasets that are re-packaged, both the original license and the license of the derived asset (if has changed) should be provided.
If this information is not available online, the authors are encouraged to reach out to the asset's creators.
13. New Assets
Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?
Answer: [Yes]
The only new asset is our code which is provided with this paper as supple- mentary material and is well documented its own and also explained in Section and in the Appendix Section
Justification:
Guidelines:
The answer NA means that the paper does not release new assets.
Researchers should communicate the details the dataset/code/mmodll as part their submissions via structured templates. This includes details about training, license, limitations, etc.
The paper should discuss whether and how consent was obtained from people whose asset used.
At submission time, remember to anonymize your assets (if applicable). You can either create anonymized URL or include anonymized zip file.
14. Crowdsourcing and Research with Human Subjects
Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well as details about compensation (if any)?
Answer: [NA]
Justification: The experiments in the paper are based on the NYU fastMRI dataset 774 which a publicily available dataset.
75
Guidelines:
The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.
Including this information in the supplemental material is fine, but if the main contribu- tion of the paper involves human subjects, then as much detail as possible should be included in the main paper.
According the NeurIPS Code of Bthies, workers involved in data collection, curation, or other labor should be paid at least the minimum wage in the country of the data collector.
Institutional Review Board (IRB) Approvals or Equivalent for Research with Human Subjects
31

---

Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or equivalent approval/review based on the requirements of your country or institution) were obtained?
Answer: [NA]
Justification: This is a theoretical paper (but with potentially huge impact in the field medical images) but all the images used for conducting our experiments are publicly available (and anonymized) and there is no need to get Institutional Review Board (IRB) Approvals or Equivalent for Research with Human Subjects.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing research with human subjects.
Depending the country in which research is conducted, IRB approval (or equivalent) may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.
We recognize that the procedures for this may vary significantly between institutions and locations, and we expect authors to adhere to the NeurlPS Code of Ethics and the guidelines for their institution.
For initial submissions, not include any information that would break anonymity (if applicable), such as the institution conducting the review.
32


Reviews:
[{"review_id": "mklKo87Rtp", "reviewer": "Reviewer_BKX7", "summary": "The paper presents a framework for constructing non-asymptotic confidence intervals around the debiased LASSO estimator. It derives a data-driven adjustment whereby the means and variances of the bias term of the debiased LASSO are estimated from the data and used to correct the confidence intervals. The framework is applied to the learned estimator from unrolled neural networks for real-world image reconstruction tasks, where the two moments are shown to be sufficient for modeling the bias term.", "questions": "- For confidence intervals with significance level $\\alpha$, the method often seems to have coverage beyond $1-\\alpha$. Is the method prone to inefficiency or overcoverage of the CIs? It would be great to see some discussions in the experiments section as to where we could gain precision, for instance from the optimization of $\\gamma$, and also refer to Section A in the main text.\n- What is meant by the \"image support?\" Could the authors please elaborate in general on what $S$ means and illustrate $S$ in the case of the MRI images for some selected $i$?\n- Why is $|W_j| \\sim {\\rm Rice}$ in L595 and not half-normal?", "limitations": "The authors acknowledge that the accuracy of the method depends on the quality of the moment estimates and the ability to minimize the length over a larger parameter set, both of which depend on the data size. They also discuss opportunities to explore higher moments and other neural net architectures.", "rating": 7, "confidence": 3, "soundness": 3, "presentation": 2, "contribution": 3, "strengths": "- The non-asymptotic treatment is a promising and worthwhile extension to the debiased LASSO that's likely to benefit a variety of high-dimensional regression applications. \n- It's a convenient plug-in method around existing estimators of the debiased LASSO.\n- The experiments include representative settings where the remainder term is significant, and the relative norm $||R||/||W||$ is quantified for each experiment.\n- The coverage levels in the experiments are convincing overall, aside from a few remaining questions (see \"Questions\")", "weaknesses": "See \"Questions\" for questions regarding the proofs and interpretation of experimental results.\n\nThe text and figure formatting could be improved for clarity:\n- Please make the figures larger. The figures are missing axis labels and/or legends. Also, the tick and axis labels are too small.\n- Please label individual subfigures in addition to describing them in the figure caption (e.g., \"(a) w/o data adjustment\" for Figure 1).\n- For subfigures 1(d) and 1(e), and similar figures throughout the text, it would be helpful to overlay the confidence level $1-\\alpha$ in a horizontal lines.\n- For Figure 3, please display (b) and (c) on the same y-axis scale.\n- L49: confusing phrasing, \"when the dimensions of the problem grow\" to describe the asymptotic setting", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We thank you for your meticulous examination of our paper and for offering valuable feedback and criticism. We are particularly thankful for suggestions to improve clarity and for acknowledging that our work is likely to benefit a variety of high-dimensional regression applications. We address the weaknesses and questions below, and for the final version, we will make the figures larger since we also have one more page available. Please see also the attached pdf with some of the figures.\n\n- W1. *Please make the figures larger. The figures are missing axis labels and/or legends. Also, the tick and axis labels are too small. Please label individual subfigures in addition to describing them in the figure caption (e.g., \"(a) w/o data adjustment\" for Figure 1). For subfigures 1(d) and 1(e), and similar figures throughout the text, it would be helpful to overlay the confidence level $1-\\alpha$ in a horizontal line. For Figure 3, please display (b) and (c) on the same y-axis scale.*\n  - Thank you for all the comments to improve the readability of our figures. The new ones are attached in the pdf. We will change all of them (and enlarge them) in the final version since we have one page more.\n\n- W2. *L49: confusing phrasing, \"when the dimensions of the problem grow\" to describe the asymptotic setting''.*\n  - With this sentence, we want to express that the dimension of the ground truth $N$, as well as the dimension of the measurements $m$, tends towards infinity with a specific ratio. This is a common assumption in the high-dimensional statistics literature. See, e.g., the papers:\n    - Javanmard, A. and Montanari, A.. Hypothesis Testing in High-Dimensional Regression\nunder the Gaussian Random Design Model: Asymptotic Theory. IEEE Trans. on Inform. Theory, 60(10):6522–6554, 2014\n    - van de Geer, S., Bühlmann, P., Ritov, Y., and Dezeure, R. (2014). On asymptotically optimal confidence regions and tests for high-dimensional models. The Annals of Statistics, 42(3).\n\n    However, in the introduction, we want to avoid explanations that are too technical. We will add a short sentence clarifying this point.\n\n- Q1. *Is the method prone to inefficiency or overcoverage of the CIs? It would be great to see some discussions in the experiments section as to where we could gain precision, for instance, from the optimization of $\\gamma$, and also refer to Section A in the main text.*\n  - That is a great question. One main advantage of our method is its generality, i.e., that it does not require assumptions on the distribution of the ground truth data except for the existence of the second moment. We handle this general case by exploiting an empirical version of Chebyshev's inequality for the remainder term, which is sharp. Hence, our approach can deal with distributions that behave badly. If you have a *nice* distribution, then our method might be prone to overcoverage. This is the price we pay for its generality. One way to overcome this trade-off is to use higher moments in the estimation process, e.g., starting with the fourth moment. Regarding the choice of $\\gamma$, we will discuss it and refer to Section A in the main text. In particular, we will provide numerics for different choices of $\\gamma$, along with a discussion about them in the experiments section. Thank you for this great suggestion.    \n\n- Q2. *What is meant by the \"image support?\" Could the authors please elaborate in general on what means and illustrate in the case of the MRI images for some selected?*\n  - Sorry, we will clarify this in the paper. The image support consists of all pixels whose value is nonzero, i.e., for $x\\in\\mathbb{C}^N$, the (image) support is $\\{i\\in [1,..., N] : x_i\\neq 0\\}$. Roughly speaking, the support of an image contains all non-black pixels.\n  \n- Q3. *Why is in $\\vert W_j\\vert \\sim\\operatorname{Rice}$ L595 and not half-normal?*\n  - Indeed, for a real normal variable, the absolute value would be half-normal. However, our method allows for a more general setting (since we could measure the phase of an MR image and get body movement detection), and we assume the variable $W_j$ to be *complex* normal, resulting in a Rice distribution for $\\vert W_j\\vert$.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "jrYvndiDJW", "reviewer": "Reviewer_9jkp", "summary": "Improve debiasing technique for better estimation/inference for high-dim models.", "questions": "Can similar to Theorem 3 results be provided for other well-known distributions? Maybe some heavy-tail dist?", "limitations": "NA", "rating": 6, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "Non-asymptotic result which helps in better numerical performance compared to asymptotic CIs.\n\nGeneral idea which can be extended to other statistical models.", "weaknesses": "NA", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thank you for your review and question about generalizing our result to other distributions. We carefully address it below. We would like to re-emphasize that our method is general enough and allows, for the first time, to quantify uncertainty when we do not have access to the ground truth and estimates of our solution (as in the case of complicated neural networks). We illustrate the method by using a SOTA network for inverse problems (It-Net by Genzel et al. (2022a)), but the method can be applied to many scenarios. That said, we would really appreciate it if the reviewer could raise the score to properly acknowledge the generality, rigor, and broad applicability of our method.\n\n- Q1. *Can similar to Theorem 3 results be provided for other well-known distributions? Maybe some heavy-tail dist?*\n  - Thanks for the excellent question. Our Theorem 3 was focused on the Gaussian case because we observed that the remainder term follows a Gaussian distribution in many MRI settings. However, Theorem 3 can indeed be generalized once the distribution of $R$ is known (even for a heavy-tailed distribution as illustrated below with a complex t-distribution), and we will include this proof and discuss how to generalize it in the final version. More precisely, we can bound the estimation error $\\vert \\hat{x}^u_j - x_j\\vert$ analogously to the proof of Theorem 2 by $\\mathbb{P}(\\vert \\hat{x}^u_j - x_j\\vert \\geq r_j(\\alpha)) = \\mathbb{P}( \\vert W_j + R_j \\vert > r_j(\\alpha)) \\leq \\mathbb{P}( \\vert W_j \\vert > r_j^W(\\alpha)) + \\mathbb{P}( \\vert R_j \\vert > r_j^R(\\alpha))$. The distribution of $\\mathbb{P}( \\vert W_j \\vert > r_j^W(\\alpha))$ is determined by the Gaussian noise and can be similarly computed as in Theorem 2. But if, instead, $R$ follows a heavy-tailed distribution, we can still compute $\\mathbb{P}( \\vert R_j \\vert > r_j^R(\\alpha))$ and choose $\\alpha$ such that $\\mathbb{P}(x_j \\in  C_j(\\alpha)) \\geq 1-\\alpha$ holds for a given radius $r_j(\\alpha)$. We just established the following theorem to illustrate our point for a certain heavy-tailed distribution.\n\n\n**Theorem** *Let $\\hat{x}^u\\in\\mathbb{C}^N$ be a debiased estimator for $x\\in\\mathbb{C}^N$ with a remainder term following a complex t-distribution with a degree of freedom $\\nu>2$, i.e. $R\\sim\\mathcal{C}t_\\nu(0,\\Sigma_R)$. Set $\\eta_j:=(\\Sigma_R)_{jj}$. Then, $C_j(\\alpha)=\\\\{ z \\in\\mathbb{C}\\mid \\vert z-\\hat{x}^u_j\\vert \\leq r_j(\\alpha)\\\\}$ with radius*\n\n\\begin{equation}\n        r_j(\\alpha) = \\frac{\\sigma(M\\hat{\\Sigma}M^\\ast)_{jj}^{1/2}}{\\sqrt{m}}\\sqrt{\\log\\left(\\frac{1}{\\gamma_j \\alpha}\\right)} + \\sqrt{\\frac{\\eta_j\\nu}{2}}\\sqrt{(1-\\gamma_j)^{-2/\\nu} \\alpha^{-2\\nu} - 1}.\n\\end{equation}\n\n*is valid, i.e. $\\mathbb{P}\\left( x_j \\in C_j(\\alpha)\\right)\\geq 1-\\alpha$.*\n\n**Proof**\nWe can bound the estimation error $\\vert \\hat{x}^u_j - x_j\\vert$ analogously to the proof of Theorem 2 by\n\\begin{align*}\n        \\mathbb{P}(\\vert \\hat{x}^u_j - x_j\\vert \\geq r_j(\\alpha)) &= \\mathbb{P}( \\vert W_j + R_j \\vert > r_j(\\alpha)) \\leq \\mathbb{P}( \\vert W_j \\vert > r_j^W(\\alpha)) + \\mathbb{P}( \\vert R_j \\vert > r_j^R(\\alpha))\n\\end{align*}\nThe distribution of $\\mathbb{P}( \\vert W_j \\vert > r_j^W(\\alpha))$ is determined by the Gaussian noise and can be computed as\n\\begin{align*}\n        r_j^W(\\alpha) = \\frac{\\sigma(M\\hat{\\Sigma}M^*)_{jj}^{1/2}}{\\sqrt{m}}\\sqrt{\\log\\left(\\frac{1}{\\gamma_j\\alpha}\\right)}\n\\end{align*}\n\nsimilarly to Theorem 2. If $R\\sim\\mathcal{C}t_\\nu(0,\\Sigma_R) $, then the marginal distribution $R_j$ is also complex t-distributed, i.e. $R_j \\sim(0,\\eta_j)$, see [OTKP12]. Moreover, the probability density function of $\\vert R_j\\vert$ is\n\n\\begin{align}\n        f(r) = \\frac{2r}{\\eta_j}\\left(1+\\frac{2r^2}{\\eta_j \\nu}\\right)^{-(\\nu/2+1)}.\n\\end{align}\n\nHence,\n\n\\begin{align}\n    \\mathbb{P}( \\vert R_j \\vert > r_j^R(\\alpha))\n    = \\int\\limits_{r_j(\\alpha)}^{\\infty} f(r) dr\n    = \\left[ -\\frac{1}{\\left(\\frac{2r^2}{\\eta_j \\nu}+1\\right)^{\\nu/2}} \\right]_{r_j(\\alpha)}^{\\infty}\n    = \\frac{1}{\\left(\\frac{2r^2}{\\eta_j \\nu}+1\\right)^{\\nu/2}}.\n\\end{align}\n\nSetting this probability equal to $(1-\\gamma_j)\\alpha$ requires\n\\begin{equation}\n    r_j^R(\\alpha) = \\sqrt{\\frac{\\eta_j \\nu}{2}}\\sqrt{(1-\\gamma_j)^{-2/\\nu} \\alpha^{-2\\nu} - 1}.\n\\end{equation}\n\nSince we assumed $\\nu>2$ the inequality $(1-\\gamma_j)^{-2/\\nu} \\alpha^{-2\\nu}>1$ holds. Combining $r_j(\\alpha) = r_j^W(\\alpha)+r_j^R(\\alpha)$ concludes the proof.\n\n\n\n[OTKP12] - Esa Ollila, David E. Tyler, Visa Koivunen, and H. Vincent Poor. Complex Elliptically\nSymmetric Distributions: Survey, New Results and Applications. IEEE Transactions on\nSignal Processing, 60(11):5597–5625, 2012", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "qUJjyeUpN0", "reviewer": "Reviewer_WkBX", "summary": "This work develops an uncertainty quantification technique based on the debiased LASSO. The error is decomposed into noise and bias terms, which allows non-asymptotic confidence intervals to be derived. An empirical version of Chebyshev's inequality allows for their construction when the bias term is only assumed to have finite second moment, while sharper estimates are obtained in the setting where it is Gaussian. Numerical examples are given.", "questions": "Can you please address the above issues?", "limitations": "Limitations are discussed adequately. As mentioned, there is no mention of conformal prediction.", "rating": 8, "confidence": 4, "soundness": 4, "presentation": 3, "contribution": 4, "strengths": "This is a good paper and in my opinion should probably be accepted.", "weaknesses": "The main weakness is that the proposed method is a competitor to conformal prediction, however there is no comparison of these methods or even mention of this. Some discussion on conformal prediction, and the relative merits of the new technique, are probably required for publication.\n\nThe figures are too small, making them hard to interpret. This is compounded by the size of the text in the images. Their presentation should be rethought and fixed.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We are particularly grateful to the reviewer for raising the insightful point regarding conformal prediction and we will add some clarification about the difference between this technique and our work. While the two methods address aspects of total uncertainty, including both epistemic and aleatoric components, we carefully explain the fundamental differences between our approach and conformal prediction in our response below. We provide a detailed discussion on how our method complements rather than competes with conformal prediction, potentially opening avenues for future research that could bridge these approaches. We believe this clarification significantly enhances the positioning and contribution of our work within the broader landscape of uncertainty quantification in ML, and we would appreciate it if the reviewer could take this into account to raise our score.\n\n- W1. *The main weakness is that the proposed method is a competitor to conformal prediction. However, there is no comparison of these methods or even mention of this...*.\n  - Thanks for the very relevant comment. We will add a discussion to the final version of the paper. Indeed, both methods produce a confidence/prediction interval for the output/prediction. However, they are inherently different approaches. Conformal prediction shines in generating prediction intervals for new observations (images) given the previous ones. On the other hand, the debiased LASSO produces precise confidence intervals for individual regression coefficients (pixels of a given image). The debiasing step corrects the bias made by the model; it is also particularly suited for problems in which the design matrix $A$, as well as the noise distribution, is known, e.g., inverse problems. Our method relies on additional samples (images $X_k$ and data $y_k$) only to calculate the distribution of the rest term $R$. This step of our method is always, as one can see in the experiments, the smallest portion of the error. Also, we don't need any calibration step that is computationally expensive in a regression setting. The distribution of the rest term is estimated by a random set of size $l$ chosen from the available data called the estimation data set. In this way, we can produce rigorous confidence intervals for each pixel of, for example, one single image.\n  \n    In contrast, conformal prediction solely relies on data already seen by the algorithm and on the distribution of the data (which does not need to be known). Still, the samples need to be identically distributed and independent or exchangeable. It is an ``online'' method using the $n$ previous samples and labels to predict the label (the training samples) and the confidence region of the next one (the test samples). For this, one calculates the non-conformity score for every new sample and decides whether the new sample lies in the prediction region or not, which is defined as\n\n    \\begin{align*}\n        \\mathbb{P}(X_{n+1} \\in \\Gamma^{\\alpha}(z_1,...,z_n,y_{n+1})) \\geq  1- \\alpha\n    \\end{align*}\n    with $\\Gamma^{\\alpha}(z_1,...,z_n,y_{n+1})$ denoting the prediction region and $z_{i} := (y_i,x_i)$ denoting the samples. Then, the method updates $\\Gamma^{\\alpha}$ at every step, also see [FZV23].\n\n\n    Recently, some works used conformal prediction to establish confidence intervals in a similar fashion to ours, e.g., *Conformal Prediction Masks: Visualizing Uncertainty in Medical Imaging* [K23]. They created conformal prediction-based uncertainty masks for imaging tasks. However, we see a few caveats: 1) computational costs -- The approach requires training an additional mask model and performing a calibration step, which adds computational overhead compared to methods that directly output uncertainty estimates. Instead, we don't need to perform any additional training, and the debiased method is fast. 2) Sensitivity to the choice of divergence measure -- the results might vary significantly depending on the chosen divergence measure, and it's not always clear which measure is most appropriate for a given task when one wants to quantify the uncertainty. Our debiased method has a very concrete metric on the uncertainty. 3) each value of the prediction mask is deﬁned independently from other values. Hence, it requires the user to specify a risk level for each pixel, which is cumbersome, especially in high-dimension. Our method does not require to define a risk level for each pixel. Although it is sufficient to define a global one, our method is flexible enough to handle pixel-wise different significance levels, if desired.\n\n    Debiased methods, we believe, may be preferred when accurate coefficient estimation and hypothesis testing are primary goals, whereas conformal prediction excels in scenarios where reliable prediction intervals (e.g., images of new patients based on images of previous patients) are crucial. Overall, we believe that the two methods are no competitors, but rather, combining the advantages of the two methods (the generality of CP with the sharpness of debiased methods) is an interesting future research direction. We will add a refined version of this discussion and a broader literature review for UQ for regression problems in the final version.\n\n[FZV23] Fontana, Matteo, Gianluca Zeni, and Simone Vantini. \"Conformal prediction: a unified review of theory and new challenges.\" Bernoulli 29.1 (2023): 1-23.\n\n[K23] Kutiel, Gilad, et al. \"Conformal prediction masks: Visualizing uncertainty in medical imaging.\" International Workshop on Trustworthy Machine Learning for Healthcare. Cham: Springer Nature Switzerland, 2023.\n\n- W2. *The figures are too small, making them hard to interpret. This is compounded by the size of the text in the images. Their presentation should be rethought and fixed.*\n  - Thanks. We will increase the size of the figures for the final version, which is possible since we will get one page more. Please check the attached pdf with some of the new figures.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "lJC8QFaJEl", "reviewer": "Reviewer_HzkU", "summary": "This paper focuses on Uncertainty Quantification (UQ) in high-dimensional regression. The authors develop a new data-driven approach that applies both to classical optimization methods, such as the LASSO (which imposes an l1l1​ penalty on the weights), and to neural networks. They address the limitations of traditional UQ techniques like the debiased LASSO, which often produce overly narrow confidence intervals due to significant bias in finite-dimensional data. The authors derive non-asymptotic confidence intervals by estimating the means and variances of bias terms from training data, thus enhancing the reliability of confidence intervals for a large class of predictors.", "questions": "See weaknesses.", "limitations": "The authors do discuss the limitations of their method, but due to the lack of clarity in the text, it is difficult to assess these limitations effectively.", "rating": 3, "confidence": 1, "soundness": 2, "presentation": 1, "contribution": 2, "strengths": "1. The paper seems to improve existing methods, though this is hard to tell (see weaknesses).", "weaknesses": "1. The paper uses non-standard notation, making it difficult to read. In Theorem 1, $b$ seems to represent the target, which is typically denoted as $y$. Additionally, the relationship between the matrix A and the vectors $x$ is unclear. The l-1​ norm in equation (1) is applied to $x$, but $x$ is also referred to as IID data in Theorem 1. Generally, the l-1​ norm is used to penalize weights, commonly denoted by $w$, $\\theta$, or $\\beta$, rather than the input data for LASSO regression.\n2. The paper does not clearly state the type of uncertainty being quantified, which could be clarified by addressing the first issue.\n3. Some acronyms are not defined (e.g., MR, ITSA, LASSO).\n4. Figure 1 is poorly presented. The images are very small with excessive white space in between, forcing readers to zoom in significantly. As a result, the caption becomes difficult to read.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We sincerely thank you for your thorough and constructive feedback. Your insights have highlighted important areas for improvement in our paper's clarity and presentation. We would like to emphasize our novel contribution (see general rebuttal) and we kindly ask you to check the other reviews to increase your appreciation for the paper.\nRegarding the notation we plan to include a comprehensive \"notation dictionary\" in our paper, bridging the gap between statistics, signal processing, and machine learning communities.\nFurthermore, we appreciate your point about uncertainty quantification and will incorporate a detailed discussion on aleatoric versus epistemic uncertainty. We want to assure you that we have carefully addressed all of your concerns in our responses below, including clarifying acronyms, improving figure presentation, and elucidating the relationships between variables in our equations. That being said, we kindly ask you to reconsider your score and raise it. We respectfully would like to suggest that a difference in notation understanding or a few small points about the presentation (one figure which is poorly presented or the lack of some acronyms definition) may not be sufficient grounds for rejecting a paper. We are very committed to improving the presentation to make it accessible to a wider audience and will be happy to answer any further questions.\n\n- W1. *The paper uses non-standard notation, making it difficult to read. In Theorem 1, $b$ seems to represent the target, which is typically denoted as $y$. Additionally, the relationship between the matrix $A$ and the vectors $x$ is unclear. The l-1 norm in equation (1) is applied to $x$, but $x$ is also referred to as IID data in Theorem 1. Generally, the l-1 norm is used to penalize weights, commonly denoted by $w$, $\\theta$, or $\\beta$, rather than the input data for LASSO regression.*\n  - Thanks for pointing this out. We will make extensive comments in the final version to make the notation very clear for the different communities reading the paper. We will substitute $b$ with $y$ as it is more common in statistics and machine learning literature. Regarding the relationship between $A$ and $x$, let us describe the problem setting. The linear model: $y = Ax^0+\\varepsilon$ consists of a matrix $A\\in\\mathbb{C}^{m\\times N}$ and *one* ground truth vector $x^0\\in\\mathbb{C}^N$. Such notation is very common in the *inverse problems* literature. One of our goals is to recover $x^0$ when we know $y, A$ and the distribution of $\\varepsilon$. *If $x^0$ is sparse* (e.g., radar images or angiographie images), one of the most common techniques in machine learning is the LASSO, i.e., we solve $\\arg\\min_x\\Vert Ax - y\\Vert_2^2 + \\Vert x\\Vert_1$. The $\\ell_1$ norm is usually used to *penalize nonzero entries*, i.e., the regression coefficients, regardless of whether the vector represents NN weights or, as in our case, the ground truth vector, e.g., an image obtained from physical measurements. This regularization induces sparsity. *If $x^0$ is not sparse*, we use deep learning to obtain a reconstruction $x$ since some architectures, like the example we've given in the paper, are state-of-the-art for certain inverse problems. In this case, the training data for the deep learning model consists of i.i.d. feature vectors $x^{(1)},..., x^{(l)}$, following the same distribution as $x^0$ (e.g., could be similar magnetic resonance images like images from the same part of the body but from different patients) and the corresponding target vectors $y^{(i)} = A x^{(i)} +\\varepsilon^{(i)}.$ We train the network to obtain a function $f$ that approximates $f(y^{(i)})\\approx x^{(i)}$ and has the property $f(y^0)\\approx x^0$. The point of our work is that *for the first time as far as we can tell*, we can quantify uncertainty componentwise very efficiently in a non-asymptotic way associated with the reconstruction of a given ground truth. The method not only comes with theoretical guarantees, but it is computationally cheap to implement since you don't need to recalculate the solution or retrain the model. We will be happy to provide further explanation here.\n\n- W2. *The paper does not clearly state the type of uncertainty being quantified, which could be clarified by addressing the first issue.*\n  - Thanks for this important question. We will add a discussion to the final version. We argue that we quantify the entire estimation uncertainty. One of the nice aspects of our method is that the decomposition of the estimation error into a Gaussian term and a remainder term allows for handling both types of uncertainty almost separately. With the Gaussian term $W$, we quantify the aleatoric uncertainty from the inherent measurement noise. The remainder term $R$ handles the epistemic uncertainty, which we quantify using a purely data-driven approach since for two different backward models (e.g., two different neural networks), it can be used to compare the estimation error of both models with respect to the ground truth. In this sense, our technique is rather a more general inferential uncertainty method.\n\n- W3. *Some acronyms are not defined (e.g., MR, ISTA, LASSO).*\n  - Thank you for pointing this out. The acronyms are all added to the paper. MR stands for *Magnetic Resonance*, the medical imaging modality, ISTA for *terative Shrinkage Thresholding Algorithm*, the most famous algorithm to solve the LASSO, and LASSO for *Least Absolute Shrinkage and Selection Operator*, the most important method for high-dimensional sparse regression.\n\n- W4. *Figure 1 is poorly presented. The images are very small with excessive white space in between, forcing readers to zoom in significantly. As a result, the caption becomes difficult to read.*\n  - Thanks, you are very correct. We will change the figures and labels/caption. Please check the pdf to see some of the new ones. Also, we will have one extra page which will allow us to have larger figures.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:
We sincerely thank the reviewers for their thoughtful and constructive feedback. We appreciate the time and effort invested in evaluating our work. We will increase the size of the figures (and expand their discussion in the appendix) and add individual labels. We would like to emphasize the three main contributions of our paper:

- We develop, for the first time, a novel non-asymptotic theory for constructing confidence intervals in high-dimensional learning. Unlike existing approaches that rely on asymptotic arguments, our finite-sample analysis explicitly accounts for the remainder term, providing rigorous guarantees without appealing to asymptotic regimes.

  We establish a general framework that extends debiasing techniques to model-based deep learning approaches for high-dimensional regression. This enables principled uncertainty quantification for estimators learned by neural networks, a capability crucial for reliable decision-making in safety-critical applications.
 - We demonstrate that the remainder term in debiased estimators can often be accurately modeled as a Gaussian distribution in real-world tasks (we use medical imaging as an example). Leveraging this finding, we derive Gaussian-adjusted confidence intervals that provide tight uncertainty estimates, enhancing the practical utility of debiased estimators in high-stakes domains.
- These contributions bridge the gap between established debiased theory and the practical applicability of uncertainty quantification methods in high-dimensional learning problems. 

We will carefully address all the questions raised by the reviewers below, including the confusion with the notation pointed out by Reviewer ``HzkU''.

Meta-review: This paper introduces a new rigorous data-driven technique to provide accurate uncertainty quantification (UQ) for a large class of high-dimensional regression models. UQ is a challenging problem, and new general approaches with strong theoretical backing are often welcomed by the community. All high-confidence reviewers provided positive assessments, with criticisms only levelled at certain aspects of the presentation (text in the figures is too small) and missing comparisons to some other competing methods. The former can be easily dealt with, and the latter was readily addressed in the author rebuttal. Accounting for these alterations in the final version, it is my assessment that this is a strong paper, and I recommend acceptance.
Decision: Accept (spotlight)