Uncertainty Quantification via Neural Posterior Principal Components
Authors: Elias Nehme
Omer Yair
Tomer Michaeli
Venue: NeurIPS 2023 poster (2023)
Keywords: Uncertainty Quantification
Inverse Problems
Probabilistic Modelling
Principal Components Analysis
Deep Learning
TL;DR: We predict posterior principal uncertainty directions using a single forward pass

Abstract: Uncertainty quantification is crucial for the deployment of image restoration models in safety-critical domains, like autonomous driving and biological imaging. To date, methods for uncertainty visualization have mainly focused on per-pixel estimates. Yet, a heatmap of per-pixel variances is typically of little practical use, as it does not capture the strong correlations between pixels. A more natural measure of uncertainty corresponds to the variances along the principal components (PCs) of the posterior distribution. Theoretically, the PCs can be computed by applying PCA on samples generated from a conditional generative model for the input image. However, this requires generating a very large number of samples at test time, which is painfully slow with the current state-of-the-art (diffusion) models. In this work, we present a method for predicting the PCs of the posterior distribution for any input image, in a single forward pass of a neural network. Our method can either wrap around a pre-trained model that was trained to minimize the mean square error (MSE), or can be trained from scratch to output both a predicted image and the posterior PCs. We showcase our method on multiple inverse problems in imaging, including denoising, inpainting, super-resolution, and biological image-to-image translation. Our method reliably conveys instance-adaptive uncertainty directions, achieving uncertainty quantification comparable with posterior samplers while being orders of magnitude faster. Code and examples are available on our [webpage](https://eliasnehme.github.io/NPPC/).

Uncertainty Quantification via Neural Posterior Principal Components
Elias Nehme
Omer Yair
Technion Israel Institute of Technology seliasne@campus technion. ac..iinn..........
Technion Israel Institute of Technology omeryair@campus techni on ac
Tomer Michaeli
Technion Israel Institute of Technology tomer m@ee. technion.c. ac il
Abstract
Uncertainty quantification crucial for the deployment of image restoration models in safety-critie critical domains, like autonomous driving and biological imaging. To date, methods for uncertainty visualization have mainly focused on per-pixel estimates. Yet, heatmap of per-pixel variances is typically of little practical use, as it does not capture the strong correlations between pixels. A more natural measure uncertainty corresponds to the variances along the principal components (PCs) of the posterior distribution. Theoretically, the PCs can be computed by applying PCA samples generated from conditional generative model for the input image. However, this requires generating a very large number samples at test time, which is painfully slow with the current state-of-the-art (diffusion) models. In this work, we present method for predicting the PCs of the posterior distribution for any imput image, in single forward pass of neural network. Our method either wrap around a pre-trained model that was trained to minimize the mean square еиrоr (MSE), and the posterior PCs. We showcase our method on multiple inverse problems in imaging, including denoising, inpainting, super-resolution. colorization, and biological maage-to-image translation. Our method reliably conveys instance- adaptive uncertainty directions, achieving uncertainty quantification comparable with posterior samplers while being orders of magnitude faster. Code and examples are available our webpage.
be trained from scratch to output both predicted image
Introduction
Reliable uncertainty quantification is central to making informed decisions when using predictive models. This especially important in domains with high stakes such as autonomous cars and biological/medical imaging, where based visual data, the system is asked to provide predictions that could influence human life. In such domains, communicating predictive uncertainty becomes necessity. In the particular case of image-to-image inverse problems, efficient visualization of predictive uncertainty is required. For example, in biological and medical image-to-image translation [7, 35, 37, 38], the predicted image as whole supposed inform a scientific discovery or affect the diagnosis of a patient. Hence, an effective form of uncertainty to consider in this case is mmmaticcaly--cordinntt
pixel variations that could alter the output image.
Currently, the majority of existing methods handle uncertainty in image-valued inverse problems by factorizing the output posterior into per-pixel marginals, in which case the strong correlations between pixels are completely ignored. This leads to per-pixel uncertainty estimates in the form
37th Conference on Neural Information Processing Systems (NcurIPS 2023).

---

Input
Porplral s.1.d.
Samples
29 Por-laxll variance
The diletution
Predction
Privdpal componerrs.
(a)
(b)
Figure 1: Comparison between per-pixel methods and NPPC. In structured output posteriors such as in image-to-image regression (left), uncertainty visualization in the form of per per-pixel variance maps (top) fail to convey semantic variations in the prediction, leaving the user with no clue regarding the different possibilities of the solution set. Our method NPPC (bottom) captures more natural measure uncertainty, by providing the user with input-adaptive PCs around the mean prediction. These then be used navigate the different possibilities in meaningful manner. e.g., in la NPPC is used to traverse the existence of uncertain cells in an image- toagee-toimme to-image translation task from biological imaging. Panel lb presents 2D motivating example illustrating the practical benefit of moving along PCs as opposed to the "standard" coordinate system (e.g., pixels in an image).
of variance heatmaps [20] or confidence intervals [2], which are of little practical use as they do not describe the joint uncertainty of different pixels. As result, such models suffer from model misspecification, absorbing inter-pixel covariates into per-pixel marginals, and presenting the user with unnecessarily inflated uncertainty estimates (see Fig. lb). For example, consider set of pixels along edge in an image, in the task of image super-resolution. It is clear that for severe upsampling factors, a model would typically be uncertain around object edges. However, since these pixels are all correlated with the same object, the underlying uncertainty is whether the entire edge is shifted jointly, and not whether individual pixels can be shifted independently.
Recently, the field of image-to-image regression has seen a surge of probabilistic methods inspired by advancements in deep generative models, such as diffusion models (DMs). The recurring theme in these approaches is the use of a deep generative prior in order to sample from the posterior distribution [8, 19, 50]. principle, using the resulting samples, possible to present the practitioner with the main modes of variations of the posterior uncertainty, for example using principal components analysis (PCA). However, these powerful posterior samplers come with a heavy computational price. Despite tremendous efforts over the past few years [27, 31, 43, 51], sampling from state-of-ther of-the-rrt methods is still unacceptably slow for many practical applications.
In this work, we bypass the need to learn a data-hungry and extremely slow conditional generative model, by directly training neural model to predict the principal components the posterior. Our method, which we term neural posterior principal components (NPPC), can wrap around any pre- trained model that was originally trained to minimize the mean square error (MSE), or alternatively can be trained to jointly predict the conditional mean alongside the posterior principal components. In particular, when used as a post-hoe method with MSE pre-trained models, NPPC is a general technique that can be transferred across datasets and model architectures seamlessly. This is because our network architecture inherits the structure and the learning parar-parseeters of the pre-trained model with only one key change: Increasing the filter count at the last layer and employing Gram-Schmin process to the output to ensure our found principal components are orthogonal by construction. NPPC is then trained on the pre-trained model residuals, to endow the model's point prediction with efficient anput-ad uncertainty visualization.
Several prior works proposed modeling the posterior a correlated Gaussian distribution. In particular, Dorta et al. [9] approximated the output precision matrix by a matrix of the form LLT, where L is lower triangular matrix with pre-determined sparsity pattern. More recently, Monteiro al. [33] and Meng et [30] approximated the covariance matrix by a sum a diagonal matrix and a low-rank one. However, unlike NPPC, both [9] and [30] are limited to very www-resoutton

---

images, because either they explicitly construct the (huge) covariance matrix during training or they capture only short-range correlations. Moreover, [33] suffers from training instability as multivariate Gaussians with both unknown mean and covariance are known to be notoriously unstable [46]. This limits its applicability to image segmentation, requiring proper background masking to avoid infinite covariance and overflow errors. Our method, on the other hand, does not construct the covariance matrix at any point. directly outputs the top eigenvectors of the covariance and trains using the objective of PCA. As result, our method is generally applicable to any inverse problem, enjoys faster and more stable training, and can handle high-resolution images as we demonstrate in Sec. 4. This is enabled by several design choices introduced to the architecture and training, including Gram- Schn-d output layer, a PCA loss function maximizing correlation with the residuals, and a Stopgrad trick that enables learning all principal directions jointly, mitigating the need for training separate models for separate PCs.
We compare our PCs to those extracted from samples from a conditional generative model as recently proposed in [4]. As we show, we obtain comparable results, but orders of magnitude faster. Finally, we showcase NPPC on multiple inverse problems in imaging showing promising results across tasks and datasets. In particular, we apply NPPC to scientific biological datasets, showing the practical benefit of our uncertainty estimates in their ability to capture output correlations.
2 Related Work
Model uncertainty Some methods attempt report uncertainty that is due to model misspecifica- tion and/or out-of-distribution data. Early work on quantification of such uncertainty in deep models has focused on Bayesian modeling by imposing distributions on model weights/feature activations [6. 17, 34]. These methods employ various techniques for approximating the posterior of the weights, including using MCMC [44]. variational inference [6, 25], Monte-Carlo Dropout [13], and Laplace approximation [36]. Our work rather focuses on data uncertainty as described in Section 3.
Per-pixel methods Some methods for predicting per-pixel variances assume a Gaussian distribu- tion and learn it by maximizing the log-likelihood [20]. This approach was later combined with hierarchical priors on likelihood parameters to infer family of distributions in a single deterministic neural network [1, 28]. Similarly, multi-modal approximations assuming a Gaussian Mixture Model per pixel have been proposed [5]. Deep Ensembles [22] and Test-time augmentations [53] have also been used to estimate uncertainty by measuring the variance of model predictions. More recently, distribution-fie methods such as quantile regression and conformal prediction [39] have taken over their counterparts with firm statistical guarantees. Specifically for image-to-image regression, work that stands out is that of Angelopoulos et al. [2]. The main shortcoming of these methods is the underlying assumption of i.idd pixels.
Distribution-free risk control Elaborate distribution- methods, such as Risk Controlling Prediction Sets (RCPS) [3], take pixel correlations into account by controlling some risk factorizing all pixels in an image (such as the false discovery rate in binary image segmentation). Recently, this approach has been deployed in image-to-image regression problems using a technique called Conformal Prediction Masks [21]. The main drawback of these methods is their inability to explore the different possible options, but rather, only impose upper and lower bounds on the possible solution set. In addition, they also require extra data split for calibration, which is not always readily available. An interesting recent work in this field [45] has proposed to control the risk of disentangled factors in the latent space of StyleGAN, and demonstrated informative uncertainty visualizations in inverse problems of facial images. However, the main drawbacks of this work are the key assumption of access to disentangled latent representations and the generalization gap from fake to real images.
Generative models and posterior samplers Generative models have been widely used to navigate prediction uncertainty, either in the form of conditional variational autoencoders [47] and conditional GANs[32], or more recently using state-of- th-the-rrt art score-based and denoising diffusion models [8, 12, 14, 19, 26, 41, 42, 48, 49, 54]. While the latter have achieved astounding results in the last two years, when aiming for high-quality samples, they remain extremely slow to sample from, despite promising results reported recent efforts [27, 31, 43, 51]. The recent Conffusion method [15] finetunes pre-trained diffusion models to output interval bounds in a single forward pass to achieve fast confidence interval prediction for posterior samplers. Nonetheless, the result is a per-pixel uncertainty map.
3

---

y
Pre-trained
PCA Loss
20
2
f(y:0)
10
(x 2)
WX
Gram-Schmidt w. Stopgrad
(v. Jt:
Neural P Posterior Principal Components
Figure 2: Method overview. Here, NPPC is demonstrated for image inpainting, where æ is the ground truth image, y is the masked input, and ē is the posterior mean prediction. Our method can wrap around pre-trained conditional mean predictors f (y:: 0), replicating their architecture with a slight modification at the output (see text). Using PCA loss on the errors (e = æ ②), we learn to predict the first K PCs of the posterior W1:-. ....... w directly in single forward pass of neural network w (y, -;; sp). On the right, we visualize the uncertainty captured by two PCs (w1, w24) around the mean prediction a.
Gaussian covariance approximation Works that are particularly related to the approach presented here are [9]. [33] and [30]. Specifically, Dorta al. [9] proposed to model the output posterior with correlated Gaussian, and approximated the precision matrix using the factorization A = LL", where L lower triangular matrix with predefined sparsity patterns capturing short-term correlations. The matrix is then learned during training using a maximum likelihood objective. Similarly, [33] and [30] suggested approximating the posterior covariance matrix using low-rank factorization with the latter involving second-order scores, However, the main drawback of [9] and [30] is their limited ability to generalize high-resolution RGB images while efficiently capturing long-range correlations [10]. Additionally, [33] is limited by training instability, limiting its applicability to image segmentation. In contrast, our method directly estimates eigenvectors of the covariance matrix without the need for explicitly storing the covariance matrix in memory during training as in [30]. This enables to seamlessly generalize arbitrary inverse problems and high- nigh-resolution RGB images with more stable and less expensive training steps.
Concurrent work A related approach was recently proposed in the concurrent work of [29], where the authors demonstrate training-free method to calculate the posterior PCs in the task of Gaussian denoising. The key advantage of our approach over this work is that we are not constrained to the task of Gaussian denoising and can handle arbitrary inverse problems,
show in experiments.
3 Neural Posterior Principal Components
We address the problem of predicting a signal æ E RRd based on measurements y E Rdd, In the context of imaging, y often represents a degraded version of x (e.g,, noisy, blurry), or a measurement of the same specimen/scene acquired by a different modality. We assume that æ and y are realizations of random vectors x and y with an unknown joint distribution p(æ, y). and that we have training matched input-output pairs independently sampled from that distribution.
set D (( yı) y:))
Many image restoration methods output a single prediction & for any given input y. A common choice is to aim for the posterior mean å E x y). which is the predictor that minimizes the MSE. However, single prediction does not convey to the user the uncertainty in the restoration. To achieve this goal, here we propose to also output the top K principal components of the posterior p(aly), ie., the top K eigenvectors of the posterior covariance [ x)(x x) ly = y). The PCs capture the main directions along which x could vary, given the input y. and thus provide the user with valuable information. However, their direct computation is computationally infeasible in high-dimensional settings. The main challenge we face is, therefore, how to obtain the posterior PCs without having to ever store even compute the entire posterior covariance matrix. Before we describe our method, let us recall the properties of PCA in the standard (unconditional) setting.
Unconditional PCA Given a set of N data points (æ:) where E RRd, the goal in PCA is to find a set of K orthogonal principal directions wi,... w wx along which the data varies the most. The resulting directions are ordered such that the variance of {w] x;) is biggest, the variance of

---

{wi z;} is the second biggest, etc. The objective function for finding these directions has multiple equivalent forms and interpretations. The one we exploit here is the iterative maximization of variance. Specifically, let X denote the centered data matrix whose rows consist of different observations x! after subtracting their column-wise mean, and let Wi...., wx denote the first K PCs. Then the Athh PC is given by
wx arg max [
[Xw]] 1l2:.
s.t. ||/||| 1, w 1 span { wk-1),
(I)
where for the first PC, the constraint is only ||w]| = 1. The variance along the h PC is given by o = + w
The challenge in posterior PCA Going back to our goal of computing the posterior PCs, let us assume for simplicity are given pre-trained conditional mean predictor o f3r 0) obtained through MSE minimization on the training set D. Let e; = a ã; denote the error of the conditional mean predictor given the ith measurement, y y: The uncertainty directions per sample yıs are the PCs of the error e; around the conditional mean estimate &.. Conceptually, this task is extremely difficult during training for every measurement yi, we have access only to single error sample e,. If we were to estimate the PCs directly, the result would be a trivial single principal direction equaling e; which is unpredictable test time. To address this challenge, here propose to harness the implicit bias of neural models and to learn these directions from a dataset of triplets D' = (a,,, Dr, } The key implicit assumption underlying our approach (and empirical risk minimization in general) is that the posterior mean Ex[ly y) and the posterior covariance E[(x - xx x) Tly y) vary smoothly with y. Hence, with the right architecture, such models can capitalize on inter-sample dependencies and properly generalize to unseen test points, by learning posterior PCs that change gracefully a function of y. This is much like models trained with MSE minimization to estimate the conditional mean EE[x y y). while being presented during training only with single output ac, for every measurement yi-
wish to capture
3.1 Naive solution: Iterative learning of PCs
Following the intuition from the previous section, we can parameterize the h-hh PC of the error using a neural network w& (y, æ; (Pk) with parameters &as, which has similar capacity to the pre-trained model fy: 0) outputting the conditional mean. This model accepts the measurement y and (optionally) the conditional mean estimate L, and outputs the htth PC of the error e.
Let di (y, (P)) be a model for predicting the first unnormalized direction, such that Given a dataset of triplets D' = {( yı, d:)};
wi (y:, æt;; (0i) = di (y:, (t;;;;;) / (y;, (e;;;;;)))) (-)ill-
we adopt the objective employed in (1), and propose to learn the parameters of the deput-dependent
first PC by minimizing
Swi (D)
Σ
W1 (y), ( e
(2)
(a0 34 ED)
Next, given the model predicting the first PC, we can train a model predict the second PC, W2 Rú; (22), by manipulating the output of model d2(y,, L;; p22. Specifically, following the approach in (1), we optimize the same loss (2), but construct the output of the model wz by removing the projection of d2 onto w1, and normalizing the result. This ensures that w w2(y:, R.; (2) I w w1 ã;; p
While in principle this approach can be iterated K times to learn the first K PCs, it has several drawbacks that make it impractical. First, it requires prolonged iterative training of K neural networks sequentially, preventing parallelization and leading to very long training times. Second, this approach is also inefficient at test time, as we need to compute K dependent forward passes. Finally, in this current formulation, different PCs have their own set of weights | and do not share parameters. This is inefficient as for a given input y and corresponding mean prediction &, is expected that the initial feature extraction stage for predicting the different PCs, would be similar. A better strategy therefore to design an architecture that outputs all PCs once.
3.2 Joint learning of PCs
To jointly learn the first K PCs of the error using a single neural network u قb sp), we introduce two key changes to the architecture inherited from the pre-trained mean estimator f(y; 0). First,
5

---

the number of filters at the output layer is multiplied by K, to accomodate the K PCs, Wi...., wK- Second, we introduce a Gram-Schmidt procedure at the output layer, making the directions satisfy the orthogonality constraint by construction. Formally, denote the non-orthonormal predicted set of directions by di...., dx. Then, we transform them be an orthonormal set w1,..., wk as
d1
w1 |d1l
d₁
-i= f= (dl we we)
wk |
1 It=1 (dl wew
k= ....K.
Note, however, that these changes do not yet guarantee proper learning of the PCs. Indeed, if we were learn the directions by naively minimizing the loss
L (D',)) =-
Σ
Σ
wk (yr, 2;: eil²,
(4)
(EL 1=1
then we would only recover them up to an orthogonal matrix. To see this, let W. denote the matrix that has W&( 2 (p) in its h column. Then the inner sum in (4) can be rewritten as ||W e, Now, it is easy to see that neither the loss nor the constraints are affected by replacing each W; with W. W,O,, for some orthogonal matrix O, Indeed, this solution satisfies the orthogonality constraint Wi W = I, and attains the same loss value as the original PCs.
Note that this rotation ambiguity did not exist in the naive approach of Sec. 3.1 because there, when finding the ktth given the preceding h pre-trained PCs, the loss term [m (y: m;; (p) T not affect the learning of wi..... - However, when attempting to learn all directions jointly, the preceding PCs receive a gradient signal from the 4t loss term as Wy a function of Wi...., W in the Gram-Schmidt procedure.
e2 did
To solve this problem and decouple the learning of the different PCs while still maintaining their orthogonality, we propose simple modification to the Bram-Schhidd procedure: Using Stopgrad for the previously derived PCs within the projection operators in (3) (see the red term in Fig. 2). This way, in each learning step the different PCs are guaranteed to be orthogonal, while solely optimizing their respective objective. This allows learning them jointly and recovering the solution of the iterative scheme in a single forward pass a neural network with shared parameters. Please see App. D.2 for validation of the equivalence between sequential and joint PC learning.
3.3 Learning variances along PCs
Recall that the variance along the htth direction corresponds to the average squared projection of the data along that direction. We can use that to output prediction of the variances {a of the PCs, by using loss that minimizes (o | e, e2) for every However, instead of adding K additional outputs to the architecture. we can encode the variances in the norms of the unnomarlized directions. To achieve this without altering the optimization objective for finding the hthh PC, again invoke the Stopgrad operator on the term jw, e, which is the current loss function value, and match the norm of the found PCs the current step to this projected variance by minimizing the loss
k-1 Σ We we
Lo (D',
Σ
Σ
I|de
1 cil2
(5)
(DL.). E.E') EDD)
With this, our prediction for of test-time is simply ||dh: Ek=l we) for quantitative validation of our estimated variances.
Please see App. D.3
3.4 Joint prediction of posterior mean
Thus far, we assumed we have access to a pre-trained posterior mean predictor & = [(y; 0), obtained through MSE minimization a dataset D {(x; y:)) Na of matched pairs. Hence, our entire derivation revolved around dataset of triplets D'' = {(m,, yi, N However, this not strictly necessary. In particular, the posterior mean predictor can be learned jointly alongside the PCs with an

---

p(ar)
{Z), 26)}
p( M1)
pla gr
6421
12-
0000
Pay
Puo Nol
Pus Vol
4102
CT
Owr
400
nans
6402
3100
0.205
10
0.0
0000
10
35
0.000 10
5000
Figure 3: Denoising samples from 2D Gaussian mixture. The left two panels depict the underlying (unknown) signal prior p(z) (blue heatmap), exemplar matched samples from the joint distribution (ar yo) ~ p(a,, y). and the resulting training D. The right two panels show the analytical posterior p(a]ly) (red heatmap) for two different test points y1 and y2 (marked as red dots) on top of the signal prior. Our estimated conditional mean (x (black star) and PCs scaled by the estimated std (dashed black ellipse) coincide with the analytical posterior mean (xll (red star), and scaled posterior PCs derived from the analytical covariance (solid orange ellipse).
additional MSE loss
Lu (D,4)
Σ
|
22112.
(6)
(2).))LE
Note that do not want the gradients of Lw and L. affect the recovered mean. Therefore, use a Stopgrad operator when inputting the error æ; - ã; to those loss functions.
3.5 Putting it all together
To summarize, given a dataset D = { };)) of matched pairs, we can learn a model that predicts the posterior mean well the posterior PCs, using the combined loss functions of eqs. (4)-(6),
Cal fn + Ln + Xghos
where X and λ are weights balancing the contributions of the different losses.
Experiments
We now illustrate NPPC on several tasks and datasets. In all experiments except for the toy example, we used variants the U-Net architecture [11, 40). The weighting factors for the losses were chosen such that all three terms roughly within order of magnitude of each other. Empirically, that ramping up the weight factors for the terms of the directions and the variances after the mean estimate started converging, stabilizes training. Full details regarding the architectures, the scheduler, and the per-task setting of λ₁ A2 are in App. A.
find
Toy examples Figure 3 demonstrates NPPC on a 2D denoising task, where sam- ples D; from a two-component Gaussian mixture model are contaminated by addi- tive white Gaussian noise to result in noisy measurements yı- To predict from y, we trained 5-layer MI.P with 256 hid- den features using MSE minimization, and around the estimated conditional mean & (x)ly
Table 1: Comparison of the Wasserstein 2-distance from the rank K Gaussian approximation of the GT posterior p(z(y) (y) 2 N(x;; Mxly Exly) (i.e., K rank Σxly)). on 5000 test samples (see text).
K=0
K
K K=6
Baseline NPPC
0.4
180.6
272.5
317.5
325.7
0.4
30.3
31.2
26.6
28.8
trained NPPC (instantiated with similar MLP) to output the two PCs
W1, W2, and their variances of, 03. As can be seen in the right two panels of Fig. 3, NPPC accurately predicts the ground-trute (GT) PCs (those are computed from the analytical expression for the poste- rior covariance; see App. B). To quantify the accuracy
recovered PCs, we computed the Wasser- stein 2-distance between a Gaussian approximation of the GT posterior p(æ (y) ~ N Exly), and an estimated Gaussian constructed by NPPC, p(aly) =N(xxly> W. W. ), where W, has the
scaled estimated PC o. we in its kth column. Compared baseline of a point mass at the estimated
7

---

C
-3
2 4
0
0-0
0-2
O- ---3
o-
--0
2-2
4
4
9 9
9 4
7
7
7
7
9
9
4
4
9
7
7
7
7
7
9
(a) Denoising
(b) Inpainting
Figure 4: MNIST denoising and inpainting. (a) Here we show the application of NPPC to extreme image denoising. On the left is the noisy measurement y. the estimated conditional mean é; and the (unknown) ground truth test image æ. On the right, the top row shows the first K = 5 predicted PCs wK, and the bottom rows show traversal of 3 standard deviations around ie for wh; Wt. At an extreme noise level of ag 1, the digit is either a "4" or a "9". (b) Here we show the result of NPPC on image inpainting from only the 8 bottom rows. The PCs reveal the digit either "7" a "9".
conditional mean, 8(x &). NPPC reduces the Wasserstein 2-distance by 100x from 4.05 to 0.04. Similarly, we also applied NPPC to 100-dimensional Gaussian mixture denoising task, and com- puted the Wasserstein 2-distance from a Gaussian distribution whose mean is the GT posterior mean and whose covariance is the best rank-K approximation of the GT posterior covariance. As clearly evident in Table 1, NPPC maintains a roughly constant distance to the analytical posterior, while the point mass baseline distance rapidly grows with K. See App. B for more details.
Handwritten digits Figure 4 demonstrates NPPC on denoising and inpainting of handwritten digits from the MNIST dataset. In the denoising task. we used noise of standard deviation og = 1, and in inpainting we used mask that covers the top 70% of the image. As can be seen, for denoising, the learned PCs capture both inter-digit and intra-digit variations, egg,, turning a "4" into a "9". Similarly, in inpainting, the learned PCs traverse the two likely modes around the mean estimate á., going from a "7" into a "9". More examples are available in App. D.
Faces To test NPPC faces, trained on the CelebA-HQ dataset using the original split inherited from celebA [24], resulting in 24183 images for training, 2993 images for validation, and 2824 images for testing. Figure 5 presents results 256 x 256 face images from the CelehA-HQ dataset. Here, we demonstrate NPPC on the task of inpainting. as well as on noisy 8x super-resolution with a box downsampling filter and a noise-level of ag = 0.05. As can be seen, the PCs generated by NPPC capture semantically meaningful uncertainty, corresponding to the eyes, mouth, and eyebrows. We also tested our approach on inpainting of the eyes area, on 4x super-resolution (noisy and noiseless), and on image colorization. More results and examples are provided in App. D.
Comparison to posterior samplers While traversal along the predicted PCs valuable for qualita- tive analysis, an important question is how our single forward pass PCs fair against vate-of-the-ar posterior samplers. To test this, we now compare NPPC to recent posterior samplers on CelebA 256 x 256. Specifically, we compare NPPC to DDRM [19], DDNM [54], RePaint [26]. and MAT [23] on the tasks of image super-rrsoluuiion/imaaannii
We use each of these methods to generate 100 samples per test image, and compute PCA on those samples. We perform comparisons over 100 test images randomly sampled from the FFHQ dataset [18]. Note that NPPC and MAT were trained on the training images of CelebA-HQ according
the original split of CelebA, whereas DDRM, DDNM, and RePaint rely on DDPM [14] as a generative prior, which was trained on the entirety of CelebA-HQ The results MSE (RMSE) || $ 2|2, and residual error magnitude le WW ell2 using the first K = 5 PCs. This is while being 100x faster than MAT, and 10 10 faster than diffusion-based methods. It is interesting to note that the principal angle between the subspace computed by NPPC and that computed by the baselines, is typically 90% Namely, they are nearly orthogonal. This can be attributed to the severe ill- ill-posedness of the tested settings, resulting in multiple different PCs with roughly the same projected variance. This is also evident in the residual eгror magnitude, suggesting that a very large number of PCs is required to handle such extreme degradations (see App. C).
reported in Table 2. As can be seen, NPPC achieves similar root
Biological image-to-image translation image-to-image translation refers to translating images from one domain into another [16]. In biological imaging, mmage-to-image translation has been
8

---

a Soo ++0000
4 +0070 ++000
(a) Inpainting
(b) Super-resolution
Figure 5: CelebA-HQ inpainting and 8X noisy super-resolution. (a) Application of NPPC to image inpainting with mask around the mouth. NPPC received the masked image and the mean prediction ét and predicted the first 5 posterior PCs wi,... W5. On the bottom, we show a traversal of 2 standard deviations around the mean estimate 4 for and wa, capturing uncertainty regarding open/closed mouth, and background vs shadow next to the jawline. (b) Application of NPPC to 8x noisy super-resolution with noise standard deviation d; 0.05. Similarly, the PC traversals on the bottom capture uncertainty of eye size and check color/position.
Table 2: Quantitative comparison of | 2|| //
WW ell2 with posterior samplers on 100 test images from FFHQ. Mean prediction and PCs were computed using 100 samples per test image, and compute is reported in neural function evaluations (NFEs).
Super-resolution
Inpainting
x noiseless
4x noisy
8x noiseless
8 noisy
Eyes
Mouth
NFEs
DDRM [19]
8 8.94/8.8
11.29/11.25 10.74/10.70
13.68/13.47 13.12/12.95 ++-
16.20/16.00 15.73/15.55 +/-
+f-
2-10a
DDNM [54]
8.43/8.88
13.27/11.24
13,30/10.37
10-10a
RePaint [26]
+
12.24/30.3
12.55/9.72
457-10"
MAT [23]
-/-
14.12/12.94
13.08/11.74
100
NPPC (Ours)
8.4/8.24
10.41/10.35
13.06/12.87
15.29/15.11
13.55/11.43
12.23/0.27
applied in several contexts, including for predicting fluorescent labels from bright-field images [35] and for predicting one fluorescent label (e.g., nuclear stainings) from another (e.g., actin stainings) [52]. However, unlike its use for artistic purposes, the use of image-to-image translation in biological imaging requires caution. Specifically, without proper uncertainty quantification, predicting cell nuclei from other fluorescent labels could lead to biased conclusions, cell counting and tracking play central roles in microscopy-basse based scientific experiments (e.g., drug testing). Here, we applied NPPC to dataset of migrating cells imaged live for 14h (1 picture every 10min) using a spinning- microscope [52]. The dataset consisted of 1753 image pairs of resolution 1024 x 1024, out of which 1748 were used for training, and 5 were used for testing following the original split by the authors. We started by training standard U-Net [11, 40) model using MSE minimization to predict nuclear stainings from actin stainings (see Fig. 6), and then trained NPPC the residuals. As show in Fig. 6, the PCs learned by NPPC convey important information to experimenters. For example, the first PC highlights the fact that accurate intensity estimation is not possible in this task, and thereby a global bias is valid uncertainty component. Furthermore, the remaining PCs reflect semantic uncertainty by adding/removing cells from the conditional mean estimate æ, thereby clearly communicating reconstruction ambiguity to the user.
5 Discussion
We proposed an approach for directly predicting posterior PCs and showed its applicability across multiple tasks and datasets. Nonetheless, our method does not come without limitations. First, as evident by both the PCs of NPPC and of posterior samplers, for severely ill-posed inverse problems a linear subspace with small number of PCs captures very little of the error. Hence, a large number of PCs is required to faithfully reconstruct the error. However, the main premise of this work was scientific imaging where scientists usually take reliable measurements, and the uncertainty is not as severe as ultimately the result should drive scientific discovery. Second, throughout this paper,

---

10
W0
3P4 We
-3
2
L + aayuy
A + ca7ywa
K
Figure 6: Biological image-to-image translation. NPPC applied to the task of translating one fluorescent label y (actin staining) to another fluorescent label ar (nuclear staining) in migrating cells (left). On the right, we show the predicted PCs at the top, and traversals along the rows. Yellow arrows point captured uncertain cells in (either deforming or being completely erased).
learned the PCs the same training set of the conditional error between training and validation could lead to biased PCs estimation as the learning process is usually stopped after some amount of overfitting (e.g., until the validation loss stagnates). This be solved by using an additional data split for "calibration" as is typically the case with distribution free methods [2. 39, 45J. Third, different inputs y; may have posteriors of different complexities, and may thus require a different number of PCs K. In our approach, K is hard-coded within the network's s architecture (it is the number of network outputs). Namely, we treat K as hyper-pan paremmereeeter that needs to be set in advance prior to training, acting as an upper bound on the number of directions the user may be interested exploring test time. However, recall that NPPC also predicts the standard deviations along each of the K PCs. These may serve to decide whether to ignore some of the PCs. Fourth, our learned PCs cover the entire image, whereas in some cases the interesting uncertainty structure could be local (e.g., cell data). In such circumstances, NPPC should either be applicd patch-win wise or the number of PCs should be sufficiently increased. Finally, our method is tailored towards capturing uncertainty and not sampling from the posterior. While provide the directed standard deviations, shifting along a certain direction does not guarantee samples along the way to be on the image data manifold. This be tackled by employing NPPC in the latent space of a powerful encoder, in which case small enough steps could lead to changes on the manifold in the output image. However, this is beyond the scope of this current work.
estimator. However, generalization
Conclusion
To conclude, in this work we proposed a technique for directly estimating the principal components of the posterior distribution using a single forward pass in a neural network. We discussed key design choices, including a Gram-Schmidt procedure with a Stopgrad operator, and a principled PCA loss function. Through extensive experiments, we validated NPPC across tasks and domains, showing its wide applicability. We showed that NPPC achieves comparable uncertainty quantification to the naive approach of applying PCA on samples generated by posterior samplers while being orders of magnitude faster. Finally, we applied NPPC to the challenging task of biological image-to-ing -m-ee translation, demonstrating its practical benefit for safety-critical applications. terms of broader impact, proper uncertainty quantification is crucial for trustworthy interpretable systems, particularly healthcare applications. Thus, a method for reporting and conveniently visualizing prediction uncertainty could support users and help them avoid making flawed decisions.
10

---

Acknowledgements
This research was partially supported by the Israel Science Foundation (grant no. 2318/22), by the Ollendorff Minerva Center, ECE faculty, Technion, and by gift from KLA. The authors gratefully acknowledge fruitful discussions with Daniel Freedman, Regev Cohen, and Rotem Mulayoff through- out this work. The authors also thank Matan Kleiner, Hila Manor, and Noa Cohen for their help with the figures.
References
[1] Alexander Amini, Wilko Schwarting, Ava Soleimany, and Daniela Rus. Deep evidential regression. Advances in Neural Information Processing Systems, 33:1 33:14927-14937, 2020.
[2] Anastasios N Angelopoulos, Amit P Kohli, Stephen Bates, Michael Jordan, Jitendra Malik, Thayer Alshaabi, Srigokul Upadhyayula, and Yaniv Romano. Image-to-image regression with distribution-free uncertainty quantification and applications in imaging. In International Conference Machine Learning. 2022.
[3] Stephen Bates, Anastasios Angelopoulos, Lihua Lei, Jitendra Malik, and Michael Jordan. Distribution-free, risk-controlling prediction sets. Journal of the ACM (JACM), 68(6):1-34, 2021.
[4] Omer Belhasin, Yaniv Romano, Daniel Freedman, Ehud Rivlin, and Michael Elad. Principal uncertainty quantification with spatial correlation for image restoration problems, arXiv preprint 2011 2023.
[5] Christopher M Bishop. Mixture density networks. 1994.
[6] Charles Blundell, Julien Cornebise, Koray Kavukcuoglu, and Daan Wierstra. Weight uncertainty in neural network. In International conference on machine learning, pages 1613-1622, PMLR, 2015.
[7] Eric M Christiansen, Samuel J Yang, D Michael Ando, Ashkan Javaherian, Gaia Skibinski, Scott Lipnick, Elliot Mount, Alison O'neil, Kevan Shah, Alicia Lee, et al. In silico labeling: predicting fluorescent labels in unlaheled images. Cell, 173(3):792-803, 2018.
[8] Hyungjin Chung, Jeongsol Kim, Michael Thompson Mccann, Mare Louis Klasky, and Jong Chul Ye. Diffusion posterior sampling for general noisy inverse problems. In The Eleventh Interna- tional Conference on Learning Representations, 2022.
[9] Garoe Dorta, Sara Vicente, Lourdes Agapito, Neill DF Camphell, and Ivor Simpson. Structured uncertainty prediction networks. Proceedings of the IEEE conference on computer vision and pattern recognition, pages 5477-5485, 2018.
2018.
[10] Garoe Dorta, Sara Vicente, Lourdes Agapito, Neill DF Campbell, and Ivor Simpson. Training vaes under structured residuals. arXiv preprint arXiv: arXiv:/804.00 1804.01050, 2018.
[11] Thorsten Falk, Dominic Mai, Robert Bensch, Özgün Çięek, Ahmed Abdulkadir, Yassine Marrakchi, Anton Böhm, Jan Deubner, Zoe Jäckel, Katharina Seiwald, et al. U-net: deep learning for cell counting, detection, and morphometry. Nature methods, 16(1):67-70, 2019.
[12] Berthy T Feng, Jamie Smith, Michael Rubinstein, Huiwen Chang, Katherine I. Bouman, and William T Freeman. Score-based diffusion models principled priors for inverse imaging. arXiv preprint arXiv:2304.1 7,7,,0023
[13] Yarin Gal and Zoubin Ghahramani. Dropout bayesian approximation: Representing model uncertainty in deep learning. In international conference on machine learning, pages 1050-1059. PMLR, 2016.
[14] Jonathan Ho, Ajay Jain, and Pieter Abbeel. Denoising diffusion probabilistic models. Advances in Neural Information Processing Systems, 33:6840-6851, 2020.

---

[15] Eliahu Horwitz and Yedid Hoshen. Conffusion: Confidence intervals for diffusion models. arXiv preprint arXiv:2211.09795 2022.
[16] Phillip Isola, Jun-Yan Zhu, Tinghui Zhou, and Alexei A Efros. Image-to-image translation with conditional adversarial networks. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 1134, 2017.
[17] Pavel Izmailov, Wesley J Maddox, Polina Kirichenko, Timur Garipov, Dmitry Vetrov, and Andrew Gordon Wilson. Subspace inference for bayesian deep learning. In Uncertainty in Artificial Intelligence, pages 1169-1179. PMLR, 2020.
(18) Tero Karras, Samuli Laine, and Timo Aila. A style-based generator architecture for generative adversarial networks. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 4401-4410, 2019. URL
[19] Bahjat Kawar, Michael Elad, Stefano Ermon, and Jiaming Song. Denoising diffusion restoration models. In Advances in Neural Information Processing Systems, 2022.
[20] Alex Kendall and Yarin Gal. What uncertainties do we need in bayesian deep learning for computer vision? Advances in neural information processing systems, 30, 2017.
[21] Gilad Kutiel, Regev Cohen, Michael Elad, Daniel Freedman, and Ehud Rivlin. Conformal prediction masks: Visualizing uncertainty in medical imaging. In ICLR 2023 Workshop on Trustworthy Machine Learning for Healthcare.
[22] Balaji Lakshminarayanan, Alexander Pritzel, and Charles Blundell. Simple and scalable predictive uncertainty estimation using deep ensembles. Advances in neural information processing systems, 30, 2017.
[23] Wenbo Li, Zhe Lin, Kun Zhou, Lu Qi, Yi Wang, and Jiaya Jia. Mat: Mask-aware transformer for large hole image inpainting. In Proceedings of the IEEE/CVE conference on computer vision and pattern recognition, pages 10758-10768, 2022.
[24] Ziwei Liu, Ping Luo, Xiaogang Wang. and Xiaoou Tang. Deep learning face attributes in the wild. In Proceedings of the IEEE International Conference on Computer Vision, pages 1730-3738 2015. URL http://mmaab.ie.cute www.ee..........c....ee.... wwt...cc//ppppee//eeee.......
[25] Christos Louizos and Max Welling. Multiplicative normalizing flows for variational bayesian neural networks. In International Conference on Machine Learning, pages 2218-220122 12222227, PMLR, 2017.
[26] Andreas Lugmayr, Martin Danelljan, Andres Romero, Fisher Yu, Radu Timofte, and Luc Van Gool. Repaint: Inpainting using denoising diffusion probabilistic models. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 11461-11471, 2022.
[27] Eric Luhman and Troy Luhman. Knowledge distillation in iterative generative models for improved sampling speed. arXiv preprint arXi::2101.02388 2021.
(28) Andrey Malinin and Mark Gales. Predictive uncertainty estimation via prior networks. Advances in neural information processing systems, 31, 2018.
[29] Hila Manor and Tomer Michaeli. On the posterior distribution in denoising: Application to uncertainty quantification. arXiv preprint arXiv:2309. /3598, 2023.
[30] Chenlin Meng, Yang Song, Wenzhe Li, and Stefano Ermon. Estimating high order gradients of the data distribution by denoising. Advances in Neural Information Processing Systems, 34: 25359-25 25369, 2021.
[31] Chenlin Meng. Ruiqi Gao, Diederik P Kingma, Stefano Ermon, Jonathan Ho, and Tim Salimans. On distillation of guided diffusion models. In NewrIPS 2022 Workshop on Score-Based Methods, 2022.
[32] Mehdi Mirza and Simon Osindero. Conditional generative adversarial nets. arXiv preprint arXiv:1411.1784, 2014.
12

---

[33] Miguel Monteiro, Loic Le Folgoc, Daniel Coelho de Castro, Nick Pawlowski, Bernardo Mar- ques, Konstantinos Kamnitsas, Mark van der Wilk, and Ben Glocker. Stochastic segmentation networks: Modelling spatially correlated aleatoric uncertainty. Advances in neural information processing systems, 33:12756-12767, 2020.
[34] Radford M Neal. Bayesian learning for neural networks, volume 118. Springer Science & Business Media, 2012.
[35] Chawin Ounkomol, Sharmishtaa Seshamami, Mary M Maleckar, Forrest Collman, and Gregory R Johnson. Label-free prediction of three-dimensione fluorescence images from transmitted-light microscopy. Nature methods, 15(1 1):17-92,,, 2018.
[36] Hippolyt Ritter, Aleksandar Botev, and David Barber. A scalable laplace approximation for neural networks. In 6th International Conference on Learning Representations, ICLR 2018- Conference Track Proceedings, volume 6. International Conference on Representation Learning. 2018.
[37] Yair Rivenson, Tairan Liu, Zhensong Wei, Yibo Zhang, Kevin de Haan, and Aydogan Ozean. Phasestain: the digital staining of label-free quantitative phase microscopy images using deep learning. Light: Science & Applications, 8(1):23, 2019.
[38] Yair Rivenson, Hongda Wang. Zhensong Wei, Kevin de Haan, Yibo Zhang, Yichen Wu, Harun Günaydın, Jonathan E Zuckerman, Thomas Chong, Anthony E Sisk, al. Virtual histological staining of unlabelled tissue- ausue--auoffuuorsceenne images via deep learning. Nature biomedical engineering, 3(6):466-477, 2019.
[39] Yaniv Romano, Evan Patterson, and Emmanuel Candes. Conformalized quantile regression. Advances in neural information processing systems, 32, 2019.
[40] Olaf Ronneberger, Philipp Fischer, and Thomas Brox. U-net: Convolutional networks for biomedical image segmentation. Medical Image Computing and Computer-Assister Intervention-MIICAl 2015: 18th International Conference, Munich, Germany, October 5-9, 2015, Proceedings, Part III 18, pages 234-241. Springer, 2015.
[41] Chitwan Saharia, William Chan, Huiwen Chang, Chris Lee, Jonathan Ho, Tim Salimans, David Fleet, and Mohammad Norouzi. Palette: image-to-image diffusion models. In ACM SIGGRAPH 2022 Conference Proceedings, pages 1-10, 2022.
[42] Chitwan Saharia, Jonathan Ho, William Chan, Tim Salimans, David Fleet, and Mohammad Norouzi. Image super-resolution via iterative refinement. IEEE Transactions Patlern Analysis and Machine Intelligence, 2022.
[43] Tim Salimans and Jonathan Ho. Progressive distillation for fast sampling of diffusion models. arXiv preprint 22X 2022.
[44] Tim Salimans, Diederik Kingma, and Max Welling, Markov chain monte carlo and variational inference: Bridging the gap. In International conference on machine learning, pages 1218-1226. PMLR, 2015.
[45] Swami Sankaranarayanan, Anastasios Nikolas Angelopoulos, Stephen Bates, Yaniv Romano, and Phillip Isola. Semantic uncertainty intervals for disentangled latent spaces. In Advances in Neural Information Processing Systems, 2022.
[46] Nicki Skafte, Martin Jørgensen, and Søren Hauberg. Reliable training and estimation of variance networks. Advances in Neural Information Processing Systems, 32, 2019.
[47] Kihyuk Sohn, Honglak Lee, and Xinchen Yan. Learning structured output representation using deep conditional generative models. Advances neural information processing systems, 28, 2015.
[48] Yang Song and Stefano Ermon Generative modeling by estimating gradients of the data distribution, Advances in neural information processing systems, 32, 2019.
13

---

[49] Yang Song, Jascha Sohl-Dickstein, Diederik P Kingma, Abhishek Kumar, Stefano Ermon, and Ben Poole. Score-based generative modeling through stochastic differential equations. arXiv preprini arXiv:201 1. 13456, 2020.
[50] Yang Song, Liyue Shen, Lei Xing, and Stefano Ermon. Solving inverse problems in medical imaging with score-based generative models. arXiv preprint arXiv:2111.08005, 2021.
[51] Yang Song, Prafulla Dhariwal, Mark Chen, and Ilya Sutskever. Consistency models. arXiv preprint arXiv:2303.01469, 2023.
[52] Lucas von Chamier, Romain Laine, Johanna Jukkala, Christoph Spahn, Daniel Krentzel, Elias Nehme, Martina Lerche, Sara Hernández-Pérez Pieta K Mattila, Eleni Karinou, et al. Democratising deep learning for microscopy with zerocostdl4mic, Nature communications, 12 (1):2276, 2021.
[53] Guotai Wang, Wenqil Li, Michael Aertsen, Jan Deprest, Sébastien Ourselin, and Tom Vercauteren. Aleatoric uncertainty estimation with test-time augmentation for medical image segmentation with convolutional neural networks. Neurocomputing, 338:34-45, 2019.
[54] Yinhuai Wang, Jiwen Yu, and Jian Zhang. Zero-shot image restoration using denoising diffusion null-space model. The Eleventh International Conference on Learning Representations, 2023,
14


Reviews:
[{"review_id": "PWPbnsxYYh", "reviewer": "Reviewer_aQW2", "summary": "This work proposes to do image recovery inference using posterior principal components. The posterior principal components are often regarded as a function of observed image, but estimating it from a single image is impossible. Therefore, this work proposes to learn the principal components directly by training a neural net on the triplets of data (observed image, ground truth and posterior mean). It learns all the principal components together using shared weights, and at the same time managed to preserve orthogonality. The experimental results show its usefulness in various tasks.", "questions": "1. Given different observations, the posterior distribution could be quite different. In this case, do you still select the same number of principal components for each observation?\n2. If you train two NNs, where the first NN output 5 components, and the second NN outputs 10 components. Are the top 5 components of the two NNs match each other?", "limitations": "yes", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "1. The paper is very well written and organized\n2. The task of the paper is very important and useful\n3. Designing and training a neural net that outputs all principal components at once and maintains othorgonality is nontrivial, and the paper solved the problem quite nicely.\n4. The experimetal results are encouraging, as the proposed method outperforms the state-of-the-art posterior samplers\n\nOverall, I think it is a solid paper, and its idea is simple and effective", "weaknesses": "There is no quality control on the predicted principal component. It is not necessarily a weakness, but it seems to be a common issue for most of deep learning based inference method.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "**There is no quality control on the predicted principal components**\n\nThanks for this important comment. Please note that as correctly mentioned in your summary, there exists no ground truth uncertainty in image restoration datasets. This is because each element in the dataset is comprised of a **single** posterior sample $\\\\mathbf{x}\\_i\\\\sim P\\_{X\\\\lvert Y}(\\\\mathbf{x}\\\\lvert\\\\mathbf{y}=\\\\mathbf{y}\\_i)$ for each observed image $\\\\mathbf{y}\\_i$. Therefore, as we stated in the paragraph \"The challenge in posterior PCA\" (l 153-166), directly computing the posterior covariance $\\\\mathbb{E}\\\\left[(\\\\mathbf{x}-\\\\hat{\\\\mathbf{x}})(\\\\mathbf{x}-\\\\hat{\\\\mathbf{x}})^\\\\top\\\\lvert\\\\mathbf{y}\\\\right]$ (or properties of the covariance such as the top $K$ eigenvectors) is practically impossible. The key implicit assumption underlying our approach (and empirical risk minimization in general) is that the posterior mean $\\\\mu(\\\\mathbf{y})=\\\\mathbb{E}[\\\\mathbf{x}\\\\lvert\\\\mathbf{y}]$ and the posterior covariance $\\\\Sigma(\\\\mathbf{y})=\\\\mathbb{E}[(\\\\mathbf{x}-\\\\mu(\\\\mathbf{y}))(\\\\mathbf{x}-\\\\mu(\\mathbf{y}))^\\\\top\\\\lvert\\\\mathbf{y}]$ vary smoothly with $\\\\mathbf{y}$. Hence, a neural network training on a dataset $\\\\mathcal{D}=\\\\left\\\\{\\\\left(\\\\mathbf{x}\\_i,\\\\mathbf{y}\\_i\\\\right)\\\\right\\\\}\\_{i=1}^{N\\_d}$, will be able to capitalize on inter-sample dependencies and learn a smooth approximation of the top $K$ posterior PCs as a function of $\\\\mathbf{y}$. Having said that, we agree that further verifying the quality of the predicted PCs may be good for reassuring the readers that our method is valid. For this purpose, following common practice (*e.g.* as done in refs [6] and [25]), we will include a controlled experiment by designing a toy model with a known posterior distribution $P\\_{X\\\\lvert Y}$ (*i.e.* where the PCs are known ground-truth functions of $\\\\mathbf{y}$). We'll use our approach to train a network to predict the PCs from a fixed dataset of single posterior samples $\\\\left(\\\\mathbf{x}\\_i \\\\sim P\\_{X\\\\lvert Y}(\\\\mathbf{x}\\\\lvert\\\\mathbf{y}=\\\\mathbf{y}\\_i), \\\\mathbf{y}\\_i\\\\right)$, and compare the result to the ground-truth PCs over a test set of $\\\\mathbf{y}$'s.\n\n**The complexity of the posterior distribution may vary across observations. Is the number of PCs fixed for every observation?**\n\nThanks for raising this point. The short answer is yes - *i.e.* the number of predicted PCs $K$ is fixed for different observations. This is because in our approach $K$ is hard-coded within the network's architecture (it is the number of network outputs). We treat $K$ as a hyper-parameter that needs to be set in advance prior to training, acting as an upper bound on the number of possible directions. However, please note that NPPC also predicts the standard deviations along each of the $K$ PCs for each input $\\\\bf{y}\\_i$. Therefore, if for a certain observation the standard deviations of some of the PCs are small, then the user can simply choose to ignore those PCs. We will touch on this point in our discussion.\n\n**If we train two NNs one outputting $K=5$ PCs and one outputting $K=10$ PCs, do the top $K=5$ PCs of both models match?**\n\nThat's a great experiment, thanks. We tested this on the Celeba-HQ dataset for the task of image colorization. The resulting first 5 PCs when training two NNs with 5/10 components were very similar (up to a flipped sign) with an average cosine similarity of 0.9 across the first 3 PCs, and  0.83 overall (please see Figures R1/R2 in the rebuttal PDF). For later PCs (*e.g.* the 4th and 5th) the similarity slightly drops as the error variance along multiple PCs is roughly the same, and hence the ordering of the PCs becomes less distinct and prune to optimization errors. To reinforce our results, we also tested the consistency across models outputting $K=1,2,3,4,5$ PCs and found they were also backward consistent. These results further validate that NPPC consistently outputs the PCs in the correct order. We will add this important finding to the appendix and refer to it in our final manuscript.\n"}, {"review_id": "UHIN04WTd3", "reviewer": "Reviewer_Dscw", "summary": "This paper proposes using a deep neural network to predict the principle components (PCs), and the associated uncertainties, of the output directly, instead of just the most likely output. This is done by proposing a PC loss that ensures PCs are unit vectors and are orthogonal. The experimental results indicate that the proposed method performs comparably or better than other methods on super-resolution and inpainting tasks, while using much less compute when predicting for new examples.", "questions": "- Could the predicted principle components and variances be compared to the ground truths for a dataset such as MNIST? It was done for an extremely simple toy dataset in the supplementary material.\n\n- Is there another way to verify the improvement in uncertainty quantification that the proposed method brings?\n\n- Could the proposed method be compared with other image-to-image translation methods of the biological imaging dataset?", "limitations": "The limitation of no guaranteed generalization performance is given. Is there a potential limitation related to choosing the number of PCs to predict? Societal impact was not addressed.", "rating": 5, "confidence": 2, "soundness": 3, "presentation": 3, "contribution": 2, "strengths": "The paper proposes, what is to my knowledge, a novel PC-based method that leads to a more computationally efficient method for super-resolution and inpainting tasks that performs comparable to, or better than, competing techniques. The proposed technique also estimates the data (i.e. aleatoric) uncertainty by learning the variances associated with each PC. Knowing the uncertainty of the predictions could be helpful in understanding the outputs of the proposed method, which is necessary for many scientific applications.", "weaknesses": "The main weakness of the paper is evaluations. The proposed method was only compared to two other methods on one dataset (Celeb-A-HQ). This comparison was not done on MNIST or the biological image-to-image translation dataset. The predicted PCs were not compared to the ground truth PCS. Also, the quality of the uncertainty quantification, a key claim of the paper, was not evaluated well.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "**The proposed method was only benchmarked on CelebA-HQ and not on MNIST/biological dataset. Could there be further comparisons on the biological dataset?**\n\nThe reason we did not compare to other techniques on the MNIST/biological datasets is that to the best of our knowledge, there exist no pre-trained posterior samplers for these datasets. In all our experiments, we intentionally used only pre-trained models optimized by the respective authors to avoid introducing implementation bias. Specifically, note that for the biological dataset, the authors of [44] used a cGAN based on pix2pix in their implementation, which is known to suffer from mode collapse. Hence, this provided single point estimates for every measurement, preventing us from comparing the PCs. Having said that, in the final manuscript, we will also include further comparisons with MAT (https://arxiv.org/abs/2203.15270) on image inpainting. To the best of our knowledge, MAT is the current SotA in image inpainting on CelebA-HQ (See paperswithcode). Our method achieved superior results with RMSE($\\\\downarrow$)/Residual Error Magnitude($\\\\downarrow$) of $10.71/9.42$ for “eyes”' and $12.86/11.21$ for “mouth” compared to $11.56/10.66$ and $13.73/12.53$ for MAT, all while being $100\\times$ faster.\n\n**The predicted PCs were not compared to the ground truth (GT) PCs. Could the predicted PCs and variances be compared to the GT on MNIST?**\n\nThanks for this important question. Please note that there exists no GT uncertainty in image restoration datasets. This is because each element in the dataset is comprised of a **single** posterior sample $\\\\mathbf{x}\\_i\\\\sim P\\_{X\\\\lvert Y}(\\\\mathbf{x}\\\\lvert\\\\mathbf{y}=\\\\mathbf{y}\\_i)$ for each observed image $\\\\mathbf{y}\\_i$. Therefore, as we stated in the paragraph \"The challenge in posterior PCA\" (l 153-166), directly computing the posterior covariance $\\\\mathbb{E}\\\\left[(\\\\mathbf{x}-\\\\hat{\\\\mathbf{x}})(\\\\mathbf{x}-\\\\hat{\\\\mathbf{x}})^\\\\top\\\\lvert\\\\mathbf{y}\\\\right]$ (or properties of the covariance such as the top $K$ eigenvectors) is practically impossible. The key implicit assumption underlying our approach (and empirical risk minimization in general) is that the posterior mean $\\\\mu(\\\\bf{y})=\\\\mathbb{E}[\\\\mathbf{x}\\\\lvert\\\\mathbf{y}]$ and the posterior covariance $\\\\Sigma(\\\\mathbf{y})=\\\\mathbb{E}[(\\\\mathbf{x}-\\\\mu(\\\\mathbf{y}))(\\\\mathbf{x}-\\\\mu(\\mathbf{y}))^\\\\top\\\\lvert\\\\mathbf{y}]$ vary smoothly with $\\\\mathbf{y}$. Hence, a neural network training on a dataset $\\\\mathcal{D}=\\\\left\\\\{\\\\left(\\\\mathbf{x}\\_i,\\\\mathbf{y}\\_i\\\\right)\\\\right\\\\}\\_{i=1}^{N\\_d}$, will be able to capitalize on inter-sample dependencies and learn a smooth approximation of the top $K$ posterior PCs as a function of $\\\\mathbf{y}$. Having said that, we agree that further verifying the quality of the predicted PCs may be good for reassuring the readers that our method is valid. For this purpose, following common practice (*e.g.* as done in refs [6,25]), we will include a controlled experiment by designing a toy model with a known posterior distribution $P\\_{X\\\\lvert Y}$ (*i.e.* where the PCs are known ground-truth functions of $\\\\mathbf{y}$). We'll use our approach to train a network to predict the PCs from a fixed dataset of single posterior samples $\\\\left(\\\\mathbf{x}\\_i \\\\sim P\\_{X\\\\lvert Y}(\\\\mathbf{x}\\\\lvert\\\\mathbf{y}=\\\\mathbf{y}\\_i), \\\\mathbf{y}\\_i\\\\right)$, and compare the result to the GT PCs over a test set of $\\\\mathbf{y}$'s.\n\n**The predicted PCs and variances were not evaluated well. Could the predicted variances be further verified?**\n\nAs mentioned above, the quality of the PCs is difficult to ascertain **directly** as there is no GT available. Similarly, for the $k^{\\\\text{th}}$ predicted variance $\\\\sigma\\_k^2$, we only have the norm of a single projected error $\\\\lvert\\\\mathbf{w}\\_k^{\\\\top} \\\\mathbf{e}\\_i\\\\rvert$ per measurement $\\\\mathbf{y}\\_i$, and hence no GT either. Please note that, unlike per-pixel methods, in our case we cannot compare the aleatoric uncertainty and the test error directly (*e.g.* RMSE vs. fraction of pixels above an uncertainty threshold), because our method does not assume pixels are independent (an incorrect assumption in images). Additionally, please note that we did evaluate the resulting PCs indirectly by measuring the Residual Error Magnitude $\\\\|\\\\mathbf{e}-\\mathbf{W}\\mathbf{W}^{\\\\top}\\\\mathbf{e}\\\\|\\_2$ which is a function of the PCs subspace. Nonetheless, in addition to this result, we further verified the predicted variances by comparing the projected test error $\\\\mathbf{w}\\_k^{\\\\top}\\\\mathbf{e}\\_i$ to the predicted variance $\\\\sigma\\_k^2$ for every test point $\\\\mathbf{y}\\_i$ (See Fig.R1/R2 in the rebuttal PDF). The results indicate that NPPC estimates the standard deviation with high accuracy (mean estimate of 0.96 vs GT of 1). We will add this quantitative validation to our final manuscript.\n\n**Is there a potential limitation related to choosing the number of PCs to predict?**\n\nThanks, an important point. Theoretically, different measurements may require a different number of PCs $K$ depending on the posterior's complexity. In NPPC, $K$ is a hyper-parameter hard-coded within the network's architecture (it is the number of network outputs) and needs to be set in advance prior to training, acting as an upper bound on the number of possible PCs. This could be computationally inefficient, however, please note that we also predict the standard deviations along each of the $K$ PCs for each input $\\\\mathbf{y}\\_i$. Therefore, if for a certain observation the standard deviations of some of the PCs are small, then the user can simply choose to ignore those PCs. We will touch on this point in our final manuscript.\n\n**Societal impact was not addressed**\n\nWe'll comment on this in the final version. In terms of broader impact, proper uncertainty quantification is crucial for trustworthy interpretable systems, particularly in healthcare applications (*e.g.* biological data)."}, {"review_id": "Vbk5RPweYq", "reviewer": "Reviewer_2qC5", "summary": "For uncertainty quantification in image recovery problems, the authors propose a way to train a neural network to produce estimates of the principle components of the posterior covariance matrix. Their approach starts with a neural network that produces the posterior mean and trains a new network (or a new prediction head on the posterior-mean network) to learn the principle components and their corresponding eigenvalues. They demonstrate their method on denoising, inpainting, super-resolution, and biological image-to-image translation problems.  They show that their method produces principle component estimates of a similar quality to those of diffusion models but thousands of times faster.", "questions": "The paper was very clear and so I have no questions.", "limitations": "Yes", "rating": 7, "confidence": 5, "soundness": 4, "presentation": 4, "contribution": 3, "strengths": "1. For image recovery, the idea of visualizing posterior uncertainty using principle components is a good one and, to my knowledge, has not been adequately explored in the literature.  The most common way to present image uncertainty information is to plot a pixel-wise variance map, but this does not show pixel dependencies, which are critical to understanding the structure of the uncertainty.  And trying to visually interpret uncertainty structure from dozens of posterior image samples is tedious and heuristic.\n\n2. The proposed method is fast at inference, since it uses only a single pass through a neural network.  This stands in contrast to Langevin/score/diffusion methods, which require generating many posterior samples, each of which can require thousands of passes through a neural network.\n\n3. The proposed method is widely applicable because it can be built on top of an existing conditional-mean (i.e., MMSE) estimation network, which are widely available.  This stands in contrast to, say, conditional normalizing flows, whose architectural constraints make them difficult to apply/tune on new applications.\n\n4. The numerical results are impressive.  The authors have tested their method on a wide range of applications, some of which involve consider large images (CelebA-HQ).  Also, their experiments suggest that their method gives similar RMSE and residual error magnitude to recent diffusion models. \n\n5. The paper is very clearly written.", "weaknesses": "1. The authors focus on a comparison to diffusion methods, which are very slow.  But modern conditional GANs and conditional normalizing flows (CNFs) can quickly generate posterior samples with performance that meets or exceeds that of recent diffusion models.  For example, a 2022 CVPR super-resolution contest (https://arxiv.org/abs/2205.05675) showed CNFs dominating other methods.  Compared to modern CNFs (e.g., https://arxiv.org/abs/2006.14200) or CGANs (e.g., https://arxiv.org/abs/2210.13389), it’s not clear that the proposed method has any speed or performance advantages.\n\n2. The proposed method is a one-trick pony, in that it generates only posterior principle components, and not high-quality image recoveries like posterior sampling methods do (e.g., diffusion, CGAN, CNF, etc.).  For example, the generated \"x+w\" face images are very blurry.  This weakness is acknowledged by the authors in Section 5.\n\n3. The proposed method seems practical for recovering only a few principle components (e.g., tens at most), whereas in some applications the uncertainty is not well described by only a few principle components.  In other words, sometimes the eigenvalues of the posterior covariance decays very slowly.  The authors acknowledge this issue in their face experiment.  The reviewer has observed, in their own work, that slow eigenvalue decay is the rule rather than the exception.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "**Advantages of the proposed method compared to fast posterior samplers such as conditional GANs/Normalizing Flows.**\n\nThanks, this is an important point. Please note that even in the case of a fast conditional generative model capable of sampling from the posterior using a single neural function evaluation (NFE), we still need 100 samples to faithfully perform PCA (*i.e.* at least 100 NFEs). This is as opposed to our method, which requires only a single NFE to output the PCs directly. Following your comment, to ensure we highlight this point, we will include in the final manuscript another comparison with MAT (https://arxiv.org/abs/2203.15270) on image inpainting. To the best of our knowledge, MAT is the current SotA in image inpainting on CelebA-HQ (see https://paperswithcode.com/sota/image-inpainting-on-celeba-hq). Our method achieved superior results with RMSE($\\downarrow$)/Residual Error Magnitude($\\downarrow$) of $10.71/9.42$ for \"eyes\" and $12.86/11.21$ for \"mouth\" compared to $11.56/10.66$ and $13.73/12.53$ for MAT, all while being $100\\times$ faster.\n\n**The proposed method only provides posterior principle components and does not generate high-quality posterior samples.**\n\nIndeed, as mentioned in the discussion (Section 5), the premise of this work was uncertainty quantification rather than posterior sampling. As you correctly noted, trying to visually interpret the uncertainty structure from dozens of posterior image samples is tedious and heuristic. Therefore, people usually resort to summarizing the samples either to per-pixel variance maps or alternatively to a few principal directions of variation. With this final goal in mind, we designed our method to output the PCs directly. In principle, this weakness could be tackled by employing NPPC in the latent space of a powerful encoder; however, this is beyond the scope of this current work.\n\n**The proposed method is only practical for a small number of principal components which might be insufficient in some applications.**\n\nIndeed, as we acknowledged in the discussion (Section 5), for severely ill-posed inverse problems, a linear subspace with a small number of PCs captures very little of the error. As you correctly point out, we did notice this to be the case for face images on the tasks of super-resolution  and inpainting. Please note, however, that on the task of image colorization presented in the supplementary (also for facial images), a small number of PCs ($K=5$) were actually able to recover large portions of the error. Hence, the practicality of predicting a few PCs to convey posterior uncertainty is eventually dataset and **task** dependent. Nonetheless, beyond a certain number of PCs, navigating the different components may become just as tedious as navigating the original posterior samples.\n\n\n\n"}, {"review_id": "Tbyt1RJtcY", "reviewer": "Reviewer_Ch4c", "summary": "This work proposes to model the full covariance matrix of aleatoric uncertainty for per-pixel image restoration tasks using a low-rank approximation. The low-rank approximation is estimated by a separate network trained end-to-end with the original predictor. The PCs, which are usually computed sequentially, are constructed in parallel using stop_gradiet operations.", "questions": "- What speaks against directly parameterizing the conditional output distribution using a multivariate gaussian with low-rank approximated covariance? As far as I know, you would also never have to explicitly evaluate the cov.\n\n- Could the authors comment on the training dynamics of the approach? How does the result compare to sequential training?\n\n- Could the authors comment on location invariance (see weaknesses)?\n\n- In l 160 mention that the error prediction model requires the right biases to work well. Can the authors elaborate on this?\n", "limitations": "N/A", "rating": 4, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 2, "strengths": "The approach is fast and, as far as I know, architecture agnostic.", "weaknesses": "- The approach requires training a separate model. \n\n- It is unclear how good parallel end-to-end training of the PCs is compared to sequential training. I could imagine that this type of approach yields pretty unstable training dynamics.\n\n- The covariance measured by this approach is over the entire image. I could image that for pixel-wise tasks there exists at least some location invariance.\n\n- Some important related work on low-rank approximations for pixel-wise aleatoric uncertainty [1] and efficient uncertainty estimation is missing (see works discussed in [2])\n\n- Minor: in eq (2) w_i is normalized to unit length whereas e_i is not. Is this intended?\n\n[1] Monteiro, M., Le Folgoc, L., Coelho de Castro, D., Pawlowski, N., Marques, B., Kamnitsas, K., van der Wilk, M. and Glocker, B., 2020. Stochastic segmentation networks: Modelling spatially correlated aleatoric uncertainty. Advances in Neural Information Processing Systems, 33, pp.12756-12767.\n\n[2] Postels, J., Segu, M., Sun, T., Sieber, L.D., Van Gool, L., Yu, F. and Tombari, F., 2022. On the practicality of deterministic epistemic uncertainty. In Proceedings of the 39th International Conference on Machine Learning (Vol. 162, pp. 17870-17909). PMLR.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "**Method proposition**\n\nOur work proposes to output the top $K$ covariance eigenvectors/PCs directly **without** assuming a low-rank (LR) structure. Although we can use the resulting PCs $\\\\bf{W}$ to construct an LR covariance approx., at no point do we use this assumption. Our work is inspired by PCA which is a non-probabilistic technique, unlike factor analysis (FA)/probabilistic PCA (PPCA) which explicitly assumes the posterior is Gaussian and approximates the full covariance matrix with $\\\\bf{\\\\Sigma}\\\\approx \\\\bf{\\\\Psi} +  \\\\bf{W}\\bf{W}^{\\\\top}$ where $\\\\bf{\\\\Psi}$ is diagonal. In general, the 2 methods are distinct; the subspaces of PCA and FA are different unless $\\\\bf{\\\\Psi}=\\\\sigma^2\\\\bf{I}$ as in PPCA.\n\n**Missing related work**\n\nThanks for bringing these refs to our attention. Ref [1*] is indeed related to Gaussian covariance approximations and we'll cite it in our final paper. Please note however that we did cite [25] that came out a year later in NeurIPS 21. This paper employs the same covariance approx. for image denoising rather than segmentation (a task more relevant to our work). As for ref [2*], this paper is focused on epistemic and not aleatoric uncertainty which is the focus of our work. The works dealing with aleatoric uncertainty in [2] are either referenced in our paper or focused on classification.\n\n**Why not parameterize the output using a Multivariate Gaussian (MVG) with low-rank approximated covariance?**\n\nThe short answer is training stability. More specifically, the only work we are aware of using the LR MVG in image restoration is ref [25]. In principle, as you suggest, one can use this approx. in conjunction with a log-likelihood loss to impose an LR MVG on the output. In this case, the loss can be indeed calculated efficiently without explicitly evaluating the full covariance thanks to the Woodbury matrix identity and the matrix determinant lemma. However, MVGs with both unknown mean and covariance are known to be notoriously unstable to train (*e.g.* see https://arxiv.org/abs/1906.03260). In fact, as we noted in Appendix A (l 66-74, Figure A3), we found it numerically unstable to even train a per-pixel Gaussian distribution using the standard loss (eq. (A2)) and consequently resorted to eq. (A1) (similar to ref [2]). Please note that also in your ref [1*] the authors reported stability issues (paragraph “caveat” on pg. 5), and resorted to early stopping before overflow errors occurred. In addition, they also encountered infinite covariance issues in background areas and addressed it using masking. Therefore, this approach is limited by training stability, especially in the case of model misspecification. In contrast, NPPC outputs the top PCs of the covariance without imposing any probabilistic assumptions on the output distribution.\n\n**The approach requires training a separate model.**\n\nPlease note that our approach enables training both the image prediction (*i.e.* the posterior mean) and the PCs jointly using a **single** model (see sections 3.4-3.5, eq. (7)), without breaking the task into two parts. However, as acknowledged in Appendix A (l 39-57, Figure A2), jointly learning the mean and the PCs led to less stable training and required more parameters+time to converge. Hence, for our CelebA-HQ/biological experiments, we focused on a two-step setting where the mean is pre-trained, and afterward, the result is wrapped around with NPPC predicting only the PCs and the variances. Please note that this limitation is not unique to NPPC, and is a standard strategy to stabilize variance prediction networks also for MVGs, *e.g.* as done in ref [6] (see also https://arxiv.org/abs/1906.03260). In fact, a similar strategy was adopted in ref [1*] that you pointed out, where even for the toy problem the authors reported stability issues and resorted to mean pre-training.\n\n**Parallel vs sequential training, and stability of training dynamics**\n\nIn our experiments, when pre-training the mean we did not experience training instability. As for joint vs sequential training, this is a very good question that warrants empirical validation. To realize sequential training without significantly altering the number of parameters used to predict the PCs, we trained multiple models with an increasing number of PCs $K=1,2,\\\\dots,5$, and compared the PCs backward across different models. The results for the task of image colorization on CelebA-HQ confirm that end-to-end training leads to approximately the same PCs as sequential training (See Figs. R1/R2 in the rebuttal PDF). Thanks for pointing this out, we will include these results in the final version.\n\n**Location invariance**\n\nIndeed, for certain tasks, there might be some location invariance. However, please note that even if the covariance structure is local (*e.g.* a Toeplitz matrix), the PCs are not necessarily localized. In our experiments, we used a fully conv. U-net, and hence our predicted PCs were equivariant to shifts in the input (*i.e.* shifted inputs -> shifted PCs). In fact, as we explained in Appendix A (l 23-27), for the biological data we learned the PCs on $64\\\\times 64$ patches cropped from the full images, as cell information tends to be local. At test time, we tested on $2\\\\times$ bigger patches as shown in Fig. 5 and Appendix C. Ultimately, in such cases NPPC should be either applied patch-wise or the number of PCs $K$ should be increased. We will include this point in our discussion.\n\n**In eq (2) $\\\\bf{w}\\_i$ is normalized to unit length whereas $\\\\bf{e}\\_i$ is not. Is this intended?**\n\nAs explained in Appendix A (l 58-65), we divide by $\\\\|\\\\bf{e}\\_i\\\\|\\_2\\^2$ to standardize the loss values across tasks and save expensive hyperparameter tuning. We will clarify this in the text.\n\n**Biases required for NPPC to work well (l 160)**\n\nThe “biases” we referred to in the text are the standard inductive/implicit biases underlying common architectures (*e.g.* convs. for images, etc). Thanks, we will clarify it.\n"}]

Author rebuttal:
The PDF includes 2 figures containing the results of experiments proposed by the reviewers.

Meta-review: The initial reviews were mixed. Some issues were identified and were addressed during the rebuttal and discussion period, leading to reviewers to increase their scores.  While one reviewer remained negative, a detailed response was provided by the authors.  I have reviewed both the original review, the authors comments and the paper and believe the concerns have been addressed.  As a result I recommend the paper be accepted. The authors should revise the manuscript to fully address the concerns and confusions raised in the review and discussion process for the final version.
Decision: Accept (poster)