Uncertainty Estimation for Computed Tomography with a Linearised Deep Image Prior
Authors: Javier Antoran
Riccardo Barbano
Johannes Leuschner
José Miguel Hernández-Lobato
Bangti Jin
Venue: Accepted by TMLR ()

Abstract: Existing deep-learning based tomographic image reconstruction methods do not provide accurate uncertainty estimates of their reconstructions, hindering their real-world deployment. This paper develops a method, termed as linearised deep image prior (DIP), to estimate the uncertainty associated with reconstructions produced by the DIP with total variation  (TV) regularisation. We endow the DIP with conjugate Gaussian-linear model type error-bars computed from a local linearisation of the neural network around its optimised parameters. To preserve conjugacy, we approximate the TV regulariser with a Gaussian surrogate. This approach provides pixel-wise uncertainty estimates and a marginal likelihood objective for hyperparameter optimisation. We demonstrate the method on synthetic data and real-measured high-resolution 2D $\mu$CT data, and show that it provides superior calibration of uncertainty estimates relative to previous probabilistic formulations of the~DIP. Our code is available at https://github.com/educating-dip/bayes_dip.

Published in Transactions on Machine Learning Research (12/2023)
Uncertainty Estimation for Computed Tomography with Linearised Deep Image Prior
Javier Antorán'
ja6660cam.ac.. uk
Deportment of Engineering, University of Combridge
Riccardo Barbano"
Srecaddo.barbanoottama
School of Computation, Information and Technology, Technical Univeristy of Munich
Johannes Leuschner
Jlusckn@@unn--rrmenn.d
Center for Industrial Mathematics, University of Bremen
José Miguel Hernández-Lobato
jmb233@camaac.uk uk
Department of Engineering, University of Cambridge
Bangti Jin
b.jin @cuhk. edu. hk
Department of Mathematics, The Chinese University of Hong Kong
Reviewed on OpenReview: https: // // openreviee. net/ for/m? FW
Abstract
Existing deep-learning based tomographic image reconstruction methods do not provide accurate uncertainty estimates of their reconstructions, hindering their real-world deployment. This paper develops a mothod, termed as linearised deep image prior (DIP), to estimate the uncertainty associated with reconstructions produced by the DIP with total variation (TV) regularisation. We endow the DIP with conjugate Gaussian-lincar model type error-bars computed from local linearisation of the neural network around its optimised parameters. To preserve conjugacy, approximate the TV regulariser with Gaussian surrogate. This approach provides pixel-wise uncertainty estimates and marginal likelihood objective for hyperparameter optimisation. We demonstrate the method on synthetic data and real- measured high-resolution 2D μCT data, and show that it provides superior calibration of uncertainty estimates relative to previous probabilistic formulations of the DIP. Our code available at httts://github. com educating weseaatinnnndi/p///ee dipp baaes_yyyss___
1 Introduction
Inverse problems in imaging aim recover an un- known image Rdd from the noisy measurement
R4
errof
maiginaa Seginalnn image
== A. + 1
(1)
where A E Rd, is a linear forward map, and ii.d. Gaussian noise, ie. ~ (0,0 12)). Many tomographic reconstruction problems take this form, e.g. computed lomography (CT). Due to the inherent. ill-posedness of the problem, e-g. dy dz, suitable regularisation prior is crucial for the successful
Figure X-ray reconstruction (501 x501 px2) of a wal- nut (left), the absolute error of its CT reconstruction (top) and pixel-wise uncertainty (bottom).
recovery of (Tikhonov & Arsenin 1977 Engl et al.
1996
Ito & Jin 2014b.
Equal contribution.

---

Published in Transactions on Machine Learning Research (12/2023)
In recent years, deep-learning based approaches have achieved outstanding performance on a wide variety
of tomographic problems (Arridge et al. 2019 Ongie et al.) 2020: Wang et al. 2020). Most deep learning
methods are supervised; they rely on large volumes of paired training data. Alas, these often fail to generalise
ont-of-distributtin (Antun et al.] 2020); small deviations from the distribution of the training data can
lead to severe reconstruction artelacts. Pathologies of this sort call for both unsupervised deep learning methods free from training data and thus mitigating hallucinatory artefacts (Bora et al. 2017 Heckel
& Hand 2019 Tölle al.] 2021] and uncertainty quantification (Kompa et al. 2021 Vasconcelos et al.
2022
informing the user about (unlreliability in reconstructions.
We focus on the deep image prior (DIP), perhaps the most widely adopted unsupervised deep learning approach (Ulyanov et al. 2018). DIP regularises the reconstructed image & by reparametrising it as the output of a deep convolutional neural network (CNN). It does not require paired training data, relying solely on the structural biases induced by the CNN architecture. The DIP has proven effective on tasks ranging from denoising and deblurring to challenging tomographic reconstructions (Liu al.
2019
Baguer al.
2020
Knopp & Grosser 2022: Darestani & Heckel 202T Gony et al. 2019 Cui et al.) 2021
Barutcu et. al.
2022
Nonetheless, the DIP only provides point reconstructions without uncertainty estimates.
In this work, we equip DIP reconstructions with reliable uncertainty estimates, which is an under-explored topic. In literature, there are two notable probabilistic reformulations of the DIP (Cheng et al. 2019 Tölle
et al., 202I), but their focus is on preventing overfitting rather than accurately estimating uncertainty, Distinctly from these, we only estimate the uncertainty associated with a specific reconstruction, instead of characterising a full posterior over all candidate images. We achieve this by computing Gaussian- linear
model type error-bars for a local linearisation of the DIP around its mode (Mackay] 1992, Khan et al. 2019 Immer et al. 2021b,, and refer to the method as linearised DIP. Linearised approaches have recently provided state-of-the-ars uncertainty estimates for supervised deep learning models (Daxberger et al.) 2021b
Unfortunately, the total variation (TV) regulariser, ubiquitous in CT reconstruction, makes inference in the lincarised DIP intractable and it docs not lend itself to standard Laplace (i.c. local Gaussian) approximations Helin et al. 2022p. We lackle this issue using predictive complexity prior (PredCP) (Nalisnick el al. 2021 to construct covariance kernels that induce properties similar to that of the TV prior while preserving Gaussian-lincar conjugacy. Finally, we discuss several techniques to scale the method to large DIP networks and high-resolution 2D images.
We showcase our approach on high-resolution CT reconstructions of real-measured 2D pCT projection data, cf. fig. D Empirically, the method's pixel-wise uncertainty estimates predict reconstruction errors more accurately than existing approaches to uncertainty estimation with the DIP. This is not at the expense of accuracy in reconstruction: the reconstruction obtained using the standard regularised DIP method (Baguer et al. 2020 is preserved the predictive mean, ensuring compatibility with advancements in DIP research.
The contributions of this work can be summarised as follows.
We propose a novel approach to bestow reconstruetions from the TV-reulaaried DIP with uncer- tainty estimates, by constructing local linear model by linearising the DIP around its optimised reconstruction and providing the model's error-bars a surrogate for those of the DIP.
We give an efficient implementation of the method, scaling up to high-resolution pCT data, and yielding far more accurate uncertainty estimation than existing probabilistic formulations of the DIP.
The rest of this paper organised as follows. Section 2 provides an extended discussion of the related work. Section 3 recalls preliminaries for the linearised DIP. Section g discusses the design of tractable Gaussian prior mimicking the TV prior. Section and section t present the linearised DIP and its efficient implementation. Section 7 presents the experimental investigations on synthetic and real-measured high-resolution JCT data. Section g concludes the article. Fully detailed derivations and additional experimental results are given in the supplementary material (SM).
Since this paper's first appcarance, the proposed method was used by Barbano et al. (022b to actively select X-ray scanning angles, resulting in a 30% reduction in angles needed obtain a given reconstruction
PSNR, and extended by Antoran et al. (023) scaling to larger problems by drawing samples with SGD.
2

---

Published in Transactions on Machine Learning Research (12/2023)
2 Related Work
2.1
Advances in the deep image prior
Since its introduction by Clyanov el. al. 22018
2020
the DIP has been improved with early stopping Wang
ct al.
2021 TV regularisation (Liu al. 2019 Baguer al. 2020 and pretraining (Barbano et al. 2022c
Knopp & Grosser 2022 Barbano al., 2023 We build upon these recent advancements by providing a
scalable method to estimate the error- bars of DIP's reconstructions. Obtaining reliable uncertainty estimates
for DIP reconstructions is a relatively unexplored topic. Building upon Carriga-Alonso et al. (2019) and
Novak et al. (019)). Cheng et al. (2019) show that in the infinite-channel limit, the DIP converges to a Gaussian process (GP). In the finile-channel regime, the authors approximate the posterior distribution over the DIP's parameters with stochastic gradient Langevin dynamics (SGLD) (Welling & Teh, 2011]. Laves ct al. [2020] and Tölle al. (2021) use factorisod Gaussian variational inference (Blundell et al. 2015 and
MC dropout (Hoo al.] 2018: Vasconcelos et al. 2022 respectively. These probabilistay (reatments of DIP primarily aim to prevent overfitting, opposed to accurately estimating uncertainty. While they deliver uncertainty estimates, their quality tends to be poor. In fact, obtaining reliable uncertainty estimates from deep-learning based approaches, like the DIP, largely remains challenging open problem (Antorán, 2019
Snoek et al. 2019; AAhukka et al. 2020 Foong et al. 2020 Barbano et al. 2022a: Antorán et al.) 2020 In the present work, we obtain uncertainty estimation by performing Bayesian inference with respect to the DIP model locally linearised around its optimised parameters. This distinct from the aforementioned approaches in that we only model local mode of the posterior distribution.
2.2 Bayesian inference in linearised neural networks
The Laplace method is first applied to deep learning in (Mackay as the best performing approach when it comes to Bayesian reasoning with neural networks (Daxberger et al.
1992
has seen a recent popularisation
2021b ab- Specifically, [Khan et al. (2019) and [mmer et al. (021b) show that the linearization step improves the quality of uncertainty estimates. [mmer et al. (2021a). Antorán et al. (2022 and Antorán et al. (2022
explore the linear model's evidence for model selection. Daxberger et al. (2021)) and Maddox et al. 2021
introduce subnetwork and finite differences approaches, respectively, for scalable inference with linearised models. Inference in the lincarised model is highly attractive compared to alternative approaches because it is post-hoc and it preserves the reconstruction obtained Uhrough the DIP optimisation as the predictive mean.
This line of work is also related the neural tangent kernel (Jacot et al. 2018 Lee et al, 2019 Novak et al, 2020 in which NNs are lincarised at initialisation.
3
Preliminaries
3.1
Total variation regularisation
The imaging problem given in eq. I admits multiple solutions consistent with the observation y. Thus, regularisation is needed for stable reconstruction. Total variation (TV) is perhaps the most well established regulariser (Rudin et al. 1992 Chambolle et al. 2010). The anisotropic TV semi-norm of an image vector R. imposes an L constraint on image gradients:
TV [iis X4415l+ Σ
Xi.s Xistil.
(2)
where X E hxw denotes the vector x reshaped into an image of height h by width and d, = h w. This Icads to the regularised reconstruction formulation
argmin C(x) with C(x) ||rr 211 XTV(2).
(3)
#ER4s
where the hyperparameter X 0 determines the strength of the regularisation relative to the fit term.
3

---

Published in Transactions on Machine Learning Research (12/2023)
3.2 Bayesian inference for inverse problems
The Bayesian framework provides consistent approach to uncertainty estimation in imaging problems (aipio & Somersalo 2005 Stuart 2010 Seeger & Nickisch 2011). The image to be recovered is treated as random variable. Instead of finding a single best reconstruction å, we aim to find a posterior distribution p(æly) that scores every candidate z E Rd according to its agreement with the observation y and prior belicf p(æ). The loss in eq- 3 can be viewed as the negative log of an unnormalised posterior, i.c. p(xly) xx expp(=4[1 (x)) and & as its mode, i.e. the marimum a posteriori (MAP) estimate. The least squares loss corresponds a Gaussian likelihood p(ylx) = -yy Ir and the TV regulariser to prior over images p(x) x exp( exp(-XTV(x)).
The posterior is obtained by updating the prior over images with the likelihood as
p(zly) = ((y)))p((/x)p(
(4)
for p(y) J p()lz)p(z)da the normalising constant, also known as the marginal likelihood (MLL), This latter quantity provides an objective for optimising hyperparameters, e-g. the regularisation strength X. The presence of different reconstructions with high probability under the posterior indicates uncertainty.
Our work partially departs from this framework in that it solely concerns itself with characterising plausible reconstructions around the mode i (Mackay) 1992p. This has two key advantages, i) tractability; the likelihood induced by NN reconstructions is strongly multi-modal, and both analytically and computationally intractable. In contrast, the posterior for the local model is Gaussian; ii) interpretablily: even we could obtain the full posterior, downstream stakeholders not versed in probability are likely to have little use for it. A single reconstruction and its pixel-wise uncertainty may be more interpretable end-users (Bhatt et al. 2021
3.3 The Deep Image Prior (DIP)
reparametrises the reconstructed image as the output of a CNN z(0) with learnable parameters E Re and a fixed input, which we have omitled from our notation for clarity. The DIP can be seen as a reparametrisation that provides a favourable structural bias towards natural images. Penalising the TV of the DIP's output avoids the need for early stopping and improves reconstruction fidelity Baguer et al., 2020). The resulting optimisation problem is given by
The DIP (Ulanov et al. 2018 2020
[Luu et al.
2019
Ô E argmin ||Ax(0) - vll2 + XTV(x(()), BER
(5)
and the recovered image is given by 8 = z0)). U-Net is the standard choice of CNN architecture Ronnebergen et al. 2015). Although the parameters 0 must be optimised separately for each new measurement y, we follow Baaraano et al., 2022c Knopp & Grosser
2022 to reduce the cost with task-agnostic pretraining.
3.4 Bayesian inference with linearised neural networks
Adopting the DIP parametrisation of the reconstructed image, in section 3.3 makes the Bayesian posterior in eq. intractable. Instead, this work only characterises the uncertainty associated with specific regularised reconstruction å, obtained via eq. (5). To this end, we take tangent linear model of the CNN
z0) around its optimised parameters (Macky
1992 Khan et al. 2019 Immer et al. 2021b),
h(0) == z0) J(0 - 8).
(6)
where J == azt(P)
le= E Rd is the Jacobian of the CNN function x0) with respect to its parameters 0 evaluated at. 0. We obtain error-bars for the DIP reconstruction =0) using h(O). For Gaussian noise and Gaussian prior 0, we have a conjugate setting; the posterior over the linearised model's reconstructions is a Gaussian N N(x; (0)), Enly): and the marginal likelihood of the lincarised model can be used to tune hyperparaneters (Mackay 1992 Immer et al. 2021a Antorán et al. 2022 Antorán et al. 2022
Computing both the posterior covariance Σzly and the marginal likelihood naively has cost Odd)). For large U-Nets, this impracticable (Daxberger et al.) 2021b). In scction 5 and section 0
we derive dual approach with a cost Odd) and detail an efficient implementation. Furthermore, when using the moon-uuarrttc) TV regulariser, conjugacy is lost. Indeed, the TV regulariser does not admit a Laplace (quadratic) approximation.
4

---

Published in Transactions on Machine Learning Research (12/2023)
4 The total variation as a conditionally Gaussian prior
First, we study the construction of tractable non-DIP-based priors for CT reconstruction. The gained understanding sheds insights into incorporating the TV-based priors into the linearised DIP framework. The regularised loss in eq. (3) can be interpreted as the negative log of an unnormalised posterior over roconstructions. In this context, the TV regulariscr corresponds to the prior
p(x) ス5² exp(- exp((ATV(xx)
where ZA = (xp(-XTV(x)) dr is its normalisation constant (the prior is improper, since constant vectors are in the null space of the derivative operator). Working with the prior p(x) is intractable since Zx does not admit a closed form. The Laplace method, which consists of a locally quadratic approximation, does not solve the issue because the second derivative of the TV regulariser zero everywhere it defined.
To enforce local smoothness in the reconstruction, we construct. a Gaussian prior Ma;; ja, Ems) with mean I Rd and covariance Ege E Rd. given by the Matern-1/2 kernel
Endiyyy = exp exp (
-iii-,,--- 1',j -- -0)
(8)
where i.j index the spatial locations of pixels of z, as in eq. o² E IR+ informs the pixel amplitude while the lengthscale parameter E RR+ determines the correlation strength between nearby pixels. The expected TV associated with our Gaussian prior is
2 and d(a, b) = +2. The hyperparameter
K En,. [TV(+)] X
co -oov--xp----))
with constant. See appendix A for derivation. Below we may omit the dependence of on (L, from the notation. For fixed pixel amplitude o², the expected reconstruction TV k is a bijection of the lengthscale l. We leverage this fact within the PredCP framework of Nalisnick et al. [021 to construct. a prior over that favours reconstructions with low expected TV
p(l) Exp((:) [e//adl
(10)
where Exp is the density of the exponential distribution. The resulting hierarchical prior over images
zll ~ (p, Enr), Exp(x) [8x/ae]
(11)
is Gaussian for fixed l, and thus the prior is conditionally conjugate Caussian-linear likelihoods. Figure shows agrecment between samples, drawn with Hamiltonian Monte Carlo, from the described TV-PredCP prior and the intractable TV prior, both qualitatively and in terms of distribution over image The TV prior produces samples with more correlated nearby pixel values than the factorised prior. The TV-PredCP prior captures this effect and produces even smoother samples, likely due to the presence of longer range correlation in the Matern-1/2 covariance.
5 The linearised DIP
In this section, we build a probabilistic model to characterise posterior reconstructions around 0, a mode of the regularised DIP objective (obtained using eq.
5 Section 5.1 describes the construction of a lincarised surrogate for the DIP reconstruction. Section 5.2 describes how to compute the surrogate model's error-bars and use them to augment the DIP reconstruction. Section discusses how include the effects of TV regularisation into the surrogate model. Finally, in section 5.4 describe a strategy to choose the surrogate model's prior hyperparameters using marginal likelihood objective.
5.1 From prior over parameters to prior over images
After training the DIP to an optimal TV-reulaaried setting â - =() using eq. we linearise the network around i by applying eq- 3 and obtain the affine-in-0 function h())) The error-bars obtained from Bayesian
5

---

Published in Transactions on Machine Learning Research (12/2023)
TV
TV-PredCP
Fact Gauss.
-in-DIP (MLL)
-nn-DIP (TV-MAP)
100
20
G52 TV-PhedP
CM2 Fact
In-OIP (MIL)
On-
KMNIST
swerage TV
Figure Samples from priors. From left to right. Plot shows a histogram of the average sample TV reporting an overlap between the TV and TV-PredCP priors. The factorised Gaussian prior results in larger TV values. Plot 2 shows an analogous histogram using samples from the linearised DIP (lin.-DIP) fitted to KMNIST image, where the hyperparancters (f, o²) have been optimised both with and without the TV-PredCP term. The TV-PredCP term in the DIP hyperparameter optimisation leads to smoother samples with less artefacts. Plots 3-5 show samples from the TV, TV-PredCP, and factorised Gaussian priors proposed in section g drawn using Hamiltonian Monte Carlo (HMC), Plots 6-7 show prior samples from the linearised DIP, which produces samples containing the structure of the KMNIST image usod train the network.
inference with h(0) will tell us about the uncertainty in a. To this end, consider the hierarchical model,
y ~ ((~NN
1e~~N0, (o) e~p() with (()) = (0) J(0- 0), (12)
where we place a Gaussian prior over the parameters 0 that, in turn, depends on the lengthscale f. Conditioned on the value of E, this a conjugate Gaussian-linear model and thus the posterior distribution over 0 has a closed Gaussian form. Learning the lengthscale l will allow us to incorporate TV constraints into the computod error-bars, ef. section 5.3. We have introduced the noise variance oy as an additional hyperparameter which will learn using the marginal likelihood (cf. section 5.4].
To provide intuition about the lincarised model, we push samples from 0 ~ N(;; 0, Eng), through h. The resulting reconstruction samples are drawn from Gaussian distribution with covariance Egs E Rd xds given by JT and are shown in fig. 2 Here, the Jacobian J introduces structure from the NN function around the linearisation point 0. It introduces features from the KMNIST character that the DIP was trained on.
5.2 Efficient posterior predictive computation
We augment the DIP reconstruction å with Gaussian predictive error- error-bars computed with the linearised model h described in eq. 12 yielding N(x; Emly). The posterior covariance Ezly is given by
E En Mo, J ATAJJ + - Eng Eay uy E
(13)
which is derived in appendix constant-in-9 terms in h do not affect the uncertainty estimates, and thus the crror-bars match those of the simple linear model JO. Importanll, eq.
B Here, Σez = Jeeed T, Day AT and Σny AE + oz1. The
(13 depends on the inverse of the observation space covariance E as opposed to the covariance over reconstructions, or parameters. Equation 13 scales as (d2 as opposed to Odd)) or (dg) for the maresstaddad--n-thllltttttt output (reconstruction) space or parameter space approaches, respectively [Immer et al. 2021b Daxberger et al. 2021a).
5.3 Incorporating TV-smoothness into our model as a prior
We impose constraints on h's error-bars, such that the model only considers low TV reconstructions as plausible. For this, we place a block-diagonal Matern-1/2 covariance Gaussian prior on the lincarised model's weights, similarly to Fortuin et al. (2021). We introduce dependencics between parameters in the same CNN convolutional filter
Boolugee = 0² exp ( d(i i'.j -dd(i-''jj
(14)
lss
where & indexes the convolutional filters in the CNN, 8y denotes Kronecker symbol, and (i.j) index the spatial locations of specific parameters within a filter. The lengthscale la regulates the filter smoothness.
6

---

Published in Transactions on Machine Learning Research (12/2023)
Intuitively, image generated from convolutions with smoother filters will present lower TV. Indeed, in appendix C we show a bijective relationship between this quantity and the filter lengthscale. The hyperparameter oy determines the marginal prior variance. Both parameters are defined per architectural block d E 21. D) in the U-Net and we write f= (f1.2....,, Lpl and x² = [of of....,., .0l The chosen U-Net architecture fully convolutional and thus eq. (14 applies to all parameters, reducing to a diagonal covariance for 1 x convolutions. U-Net diagram highlighting these prior blocks in fig. E
To enforce TV-smoothness,
adopt the strategy given in section 1 Since choosing large l enforces
smoothness in the output, prior placed over the filter lengthscales l can act as surrogate for the TV prior.
To make this connection explicit, we construct a TV-PredCP Nalisnicc et al. 2021
D = II Exp(Ha)
D ((e)
Orid dld
p(0)
(15)
d=1
with Ki == I8
8(0, 8())-0)) [ATV()(0)
(16)
II
being the expected TV of the CNN output over the prior uncertainty in the parameters of block d when all other entries of 0 are fixed to 0.
We relate the expected TV Kg to the filter lengthscale
la via the change of variables formula. The indepen- dence across blocks of p(() ensures dimensionality preservation, formally needed in changing variables. It follows from the triangle inequality that Ea Ka is an upper bound on the expectation under the distribution (8
(4s.
Up (fux.
Out Txi
Down (fa.ef)
cf. appendix C (TV()()))) (T)))))))
Up
(Mro, ffa)
Note that eq. 15 can be computed analytically. However, its direct computation is costly and we instead rely on numerical methods described in sec-
Down (fa. <3)
Up
3x3 conv
leally
norm
Skip ixxia
(Ko,
Down (Fa, o
group norm
tion In fig. 2 (cf. plot 2 and plots 6-7), we show samples from N (a;; Enz) where é is chosen using the marginal likelihood with TV-PredCP constraints (ef. also section 5.4p. Incorporating the TV-PredCI leads to smoother samples with less discontinuities.
Saip oba
Up 2 (G.
bilineur upsampling
Down 4 (Er, o
layer norm 2a (2x3 comy group ReLU)
come
Down (5.
Up (fr.08)
group norm
comy
Figure 3: A schematic of the U-Net architecture used the numerical experiments on Walnut data (see sec- tion 7.2 For KMNIST, we use reduced, 3-scale U-Net without group norm layers (sce fig. 20 Each light-blue rectangle corresponds to a multi-channel fea- ture map. We highlight the architectural components corresponding to each block 1,..., rate prior is defined with red and yellow boxes.
5.4 Type-11 MAP learning of hyperparameters
The calibration of the predictive Gaussian error- bars depends on the choice of the hyperparameters (o 0², E) of the hicrarchical model in eq. 12 (An- torán et al. 2022). For a given L, Gaussian-lineaa conjugacy yields a closed form marginal likelihood objective to learn the hyperparameters. In turn, to learn L, we combine the above objective with the TV-PredCP's bog-density, which acts as regulariser. The resulting expression resembles Type-II MAP
....., D for which sepa-
Rasmussen & Williams 2005) objective
log p(yle; 0²) + logp(e;00)
1 200 30 Ax
(0)1
2 1 Ddd (1.
1
log Egul Σ Eu(le 02) + log
@na(l, He
B, (17)
where B is independent of (o2 0²,() and the vector 0, E RRde is the posterior mean of the linear model's parameters. See appendix B for the detailed derivation. The bottlenock ovaluating eq. (17) is the log- determinant log |Eyyl of Eyg. which has cost O(d)). We go on describe scalable ways to approximate the log-determinant. and other costly quantities required for prediction.
7

---

Published in Transactions on Machine Learning Research (12/2023)
6 Towards scalable computation
In typical tomography setting, the dimensionality du of the image & and dy of the obscrvation can be large, e.g, dz > le5 and dy > le3. Thus holding the input space covariance matrices (e.g. End and Exly) in memory is infeasible. The latter complicates computing log | in eq. 17 (or its gradients), and its inverse in Maly: cf. eq. (33, which scale as O(dy) and O(dady) respectively. To scale the approach, we only access Jacobian and covariance matrices through matrix-vector vector products (matvecs), i.e. products resembling v Ers and " Egy for va E d and By E Rdy, We compute E RRd, through successive matvecs as
by E Eny - u. [AJEEoo (AJEnoJ'A' T
(18)
and we compute U2 Eez similarly. We compute Jacobian vector products vig JT for up E Re using forward mode automatic differentiation (AD) and v. J using backward mode AD, both with the functorch library (He & Zou 202T). We compute products with Ege by exploiting its block diagonal structure. All these operations be batched using modern numerical libraries and GPUs.
6.1 Conjugate gradient log-determinant gradients
For the Type-IT MAP optimisation in eq. 17 we estimate the gradients of log Eny parameters of interest using the stochastic trace estimator (Gibbs MacKay
with respect to the Gardner et al. 2018
1996
Ölog Evel Tr 06
0 dó
Ey H p 1 0
E
(19)
dΦ
where P is a preconditioner matrix. We approximately solve the linear system v Myy for batches of probe vectors using the GPyTorch preconditioned conjugate gradient (PCG) implementation (Dong et al. 2017
The preconditioner P is constructed using r-rank randomised SVD, by approximating AJE A ÚÄOT,
using randomised eigendecomposstionn algorithm (Halko et al. 2011 Martinsson & Tropp 2020 with
O E Xr and r = 200 < dy. The algorithm is described in detail in appendix hyperparameters 6, we interweave the updates of with the optimisation of eq. 17
Since P depends on the
6.2 Ancestral sampling for TV-PredCP optimisation
For large images, exact evaluation of the expected TV with eq. 16 is intractable. Instead, we estimate the gradient of Ka with respect to o = (o²,e) using a Monte- approximation
dr. 06
OTV(2) 00. 06
Ja
(20)
Ōx
where Ju Oz(0)
0TV(z) is evaluated at the sample z=0a and 00, is the reparamctrisation gradient 06. 0, m. 8z
de for Oa, prior sample of the weights of CNN block d. Since the second derivative of the TV semi-norm is almost everywhere zero, the gradient for the change of variables volume ratio is
82 kdd -Bu-N((uNegee) d02
(TV(2)) 80s d62 O6
Ja
(21)
6.3 Posterior covariance matrix estimation by sampling
The covariance matrix Ezly is too large to fit into memory for high-resolution tomographic reconstructions. Instead, we follow Wilson et al. (2021) in drawing samples from NNx 0, Exly) via Matheron's 's rule
Ezly 2g + Exy E Aza); To J0o: A. N/0, Ego):
(~~N00,0²))
(22)
The biggest cost lies in constructing Egg, which achieved by applying eq.
18 to the standard basis vectors
Eny = (e1, 62, ... ed,)] Egy- We then perform its Cholesky factorisation intermediate step towards matrix

---

Published in Transactions on Machine Learning Research (12/2023)
inversion, both relatively costly operations. Fortunately, we only have to repeat these once, after which the sampling step in eq. [22] can be evaluated cheaply. Alternatively as in eq- 19), we can compute the solution of the linear system, Eyy", for any By via PCG, without explicitly assembling (and thus storing in memory) the measurement covariance matrix, or computing its Cholesky factorisation. This approach allows us to scale the sampling operation large measurement spaces, where the matrix Eny may not fit in memory.
Since only nearby pixels of the predictions are expected be correlated, we estimate cross covariances for patches of only up to 10 x 10 adjacent pixcls. Using larger patches yields improvements. We use the diag(z,)² + zzz]]] for (zz) samples from stabilised formulation of Maddox et al. (2019): Exry 2k ريك
the posterior predictive distribution over a patch. Note that the samples from eq. (2 are zero mean.
6.4 Faster low-rank Jacobian matvecs
Table shows that the Jacobian matvecs- implemented through forward and backward mode AD required for sampling from the posterior pre- dictive (that is 2x ug JJ and 1x J) take x 100 % of this step's computation time h2.4h). To accolerate sampling, we construct a low-rank approximation of the Jacobian J, which we store in memory. We compute vi J and v J via matvec, as opposed to AD. This allows for fast approximation of uy Eny by substituting into eq. (18). This brings the time nceded for sampling from the posterior predictive
Table 1: Wall-clock time on an A100 GPU for the different steps of our algorithm when applied to high- resolution CT (details in section 7.2 Computations reported below the dotted line are double precision. The time taken by Jacobian matvecs during sampling is given in parenthesis.
wall-clock time
DIP optim. (after protriningg Harbano et Hyperporam. optim. (MLL) Hyperparam. optim. (TVM))
2022c
<0.1h
26.2h
35.4h
Assembble Eap
Draw 4096 poeterior eumples
2.4h
down from 2.4 hours to less than minute. We con-
(Fmiluate 4006 times o7 JTJT 1x (T)
2.4h
struct j similarly to the low-rank preconditioner P
Draw 4096 posterior samples (J & PCG)
(Fenimate 4006 times of J 1x J)
0.11
0.3h
(see section 6.1 and appendix D That is, following
Halko et al. (2011). b, we build rank-r approximation to J, by accessing only to matvees with J and JT.
While offering a well-calibrated alternative to uncertainty quantification within the DIP framework, it incurs computational overhead (see table D when compared to MC dropout, which only require a forward pass
through the network to generate a single sample.
Algorithm
1 summarises image reconstruction and uncertainty estimation with the linearised DIP.
Algorithm 1: Linearised deep image prior (lin.-DIP) inference
Inputs: noisy measurements y. CNN xz-), probabilistic model's hyperparameters, whether to use fast approximate posterior sampling fast_ samp ling
0 fit fit__IPtt DIP // by minimising eq
(5)
0, - find linearised MAP(y, x()) // using Algorithm 1 from Antorán et al. (022)
oi (a (a) + _aiiniis_hyerppraaeee (y, x00), // by maximising eq. (17) with
estimators eqs. 19) to (21) and solving linear systems with PCG
4 if nol fast_sasaaaa lin then
Eny +-
(rr))) ( (o (a) // by applying eq. (18) to rows of Ia,
Ezlp +
(03, 42) Eny) // using eq. (22
else
j +- construct construct__owrankk Coorrank_Jacobian(((0))) // by randomised SVD, cf section 6.4
Salp + pest__samlling(/// of (0z f) // using eq. (22) with and PCG
Output: mean reconstruction 20)), posterior covariance estimate E211
9

---

Published in Transactions on Machine Learning Research (12/2023)
7 Experiments
Here, we experimentally evaluate: i) the properties of the models and priors discussed in sections and and whether they lead to accurale reconstructions and calibrated uncerlainly; ii) the fidelity of the approximations described in section G and iii) the performance of the proposed method linearised DIP (lin.-DIP) relative to the previous MC dropout (MCDO) based probabilistic formulation of DIP (Laves ot al.) 2020 We attempted
to include DIP-SGLD (Cheng et al. 2019] in our analysis, but were unable to get the method to produce
competitive results on tomographic reconstruction problems. For each individual image to be reconstructed, we employ the following linearised DIP inference procedure: i) optimise the DIP weights via eq.
F
obtaining
& = =0);; ii) oplimise prior hyperparameters (oy) f, o²) via eq. (17 1 iii) assemble and Cholesky decompose
Eny with eq. [18 (this step can be accelerated using approximate methods sections 6.3 and 6.4 iv) compute
posterior covariance matrices either via eq. or estimate them via eq. 22 ; cf. Algorithm 1.
7.1 Small scale ablation analysis: reconstruction of KMNIST digits
The initial analysis uses simulated CT data obtained by applying eq. I to 50 images from the test set of the Kuzushiji-MNIST (KMNIST) dataset: 28 x 28 (dg = 784) grayscale images of Hiragana characters (Clanuwat 2018 We choose the noise standard deviation to be either 5% or 10% of the mean of Ax, denoted as (5%% or n (10%). The forward map A is discrete Radon transform, assembled via ODL (Adler et al. 2017 We use a U-Net with 3 scales and 76905 parameters (a down-sized net compared to the one in fig.
et al.
7.1.1 Comparing linearised DIP with network-free priors
We first evaluate the priors in section i.e. TV prior, TV-PredCP with a Matern-1/2 kernel, and a factorised Gaussian prior, and perform inference in the setting where the map A collects 5 angles (dy = 205) sampled uniformly from 0" to 180% and is applicd to 50 KMNIST test set images. Here, 10% noise is added. This results in very ill-posed reconstruction problem, maximising the relevance of the prior. We select the og and X hyperparametors for the factorised Gaussian prior and the TV prior respectively such that the posterior mean's PSNR is maximised across a validation set of 10 images from the K.MNIST training set. We keep the choice of o² and À hyperparameters from the first two models for our experiments with the third model: Matern-1/2 with TV-PredCP prior over E. For all priors, we perform inference with the NUTS HMC sampler. We run 5 independent chains for each image. We burn these in for 3 10 steps each and then proceed to draw 10a samples with thinning factor of 2.
Table 2: Quantitative results for inference with the
different priors introduced in section $ We report both the PSNR of B[x|y], which denotes the posterior mean reconstruction, and the PSNR of λt which denoles the posterior mode found through optimisation.
likellood
E[ris]
Fact. Gauss. TV
0.30 +: 0.17
16.15 4: 0.38
14.89 + 0.38
0.49 0.14
16.32
0.38
16.29 + 0.41
TV-PredCP
0.65 0.12
16.55
0.39
17.48 + 0.39
IDP (MLL)
1.63 0.08
19.46 + 0.52
DiP. (TV-MAP)
1.63 + 0.09
19.46 + 0.52
We evaluate test log- likelihoehod using Gaussian Kernel Density Estimation (KDE) (Silverman 1986). The kernel bandwidth is chosen using cross-validation on 10 images from the training set. The results in table 2 show that the TV-PredCP performs best in terms of the test log-likelihodd and both posterior mean and posterior mode PSNR, followed by the TV and then the factorised Gaussian. This somewhat surprising considering that this prior was designed as an approximation the intractable TV prior. We hypothesise that this may be due to the Matern model allowing for faster transitions in the image than the TV prior, while still capturing local correlations, shown qualitatively in fig. 2 This property may be well-suited to the KMNIST datasets, where most pixels either present large amplitudes or are close to 0. Dap-based predictions provide 2dB higher PSNR reconstructions than the DIP based priors, thus linearised DIP handily obtains a better test log- lig-licel than the moro-traditional methods.
7.1.2 Comparing calibration with DIP uncertainty quantification baselines
Using KMNIST, we construct test cases of different ill-posodness by simulating the observation y with four different angle sub-sampling settings for the linear operator A: 30 (dy=1230), 20 (dy=820), 10 (d,=410) and 5 (dy=)05) angles are taken uniformly from the range 09 to 180 We consider two noise configurations by
10

---

Published in Transactions on Machine Learning Research (12/2023)
stó-dev
marginal std-dev
10
0.45
10
1002
MILL QUL: 2.15)
vod
Me QU1 2.21)
lin. lin.-DIP
TV-KAP
demsity
10
0..00
10l
-..5
10"
0.00
10-2
0.3
DIP-MCDO
10P
denstt
10'
100
0.1
10-4
0.0
PENR: 24.2 SSIM: 0.963
Figure 4: Exemplary character recovered from y (using angles and (7(5)) with liP.-DIP and DIP-MCDO along with respective uncertainty estimates. lin.-DIP provides vastly improved uncertainty calibration. For lin.-DIP, the colour-map is shared between f x and sid-dev, and TV-MAP refers to Type-II MAP optimisation of hyperparameters.
adding either 5% or 10% noise to the exact data Ax. We evaluate all DIP-based methods using the same 50 randomly chosen KMNIST test set images. To ensure a best-case showing the methods, we choose appropriate hyperparameters for each number of angles and white noise percentage setting by applying grid-search cross-validation, using 50 images from the KMNTST training dataset. Specifically, we tune the TV strength À and the number of optimisation iterations for the DIP. Due to the reduced image size, we apply lincarised DIP as in section 5 without approximate computations. As an ablation study, we include additional baselines: linearised DIP without the TV-PredCP prior over hyperparameters (labelled MLL), and DIP reconstruction with a simple Gaussian noise model consisting of the back-projected observation noise N(I; A I), with x = TTTA((A A)))d=² where -1 (labelled -1- Note that non-dropout methods share the same DIP parameters 0, and thus the same mean reconstruction. Hence, higher values in log-density indicate better uncertainty calibration, i.e. the predictive standard deviation better matches the empirical reconstruction error. DIP-MCDO does not provide an explicit likelihood function over the reconstructed image. We model its uncertainty with Gaussian predictive distribution with covariance estimated from 214 samples. MNIST images are quantised to 256 bins, but our models make predictions over continuous pixel values. Thus, we simulate a de-quantisatton of KMNIST images by adding a noise jitter term of variance approximately matching that of uniform distribution over the quantisation step (Hoogeboom et al. 2020
Table 3: Mcan and std-orr of test log- liceliodd computed over 50 KMNIST test images.
(89))
#angles
10
30
(10%)
#angles
10
20
DIP (o]
0.65
0.14
1.57 1 0.02
1.55 0.02
2.02 1 0.02
DIP (o] 1)
0.27 1 0.17
1.31 1 0.01
1.62 2 0.03
1.76 1 0.94
DIP-MCDO
0.74 2 0.18
1.60
1 0.02
1.87
0.02
205 : 0.02
MCPO
0.42 0.14
1.39
001
1.70= 0.03
1885 0.04
DIn (MIL)
1.90
0.14
2.94 0.10
INP (MLL)
1.63 0.08
0.07
2.48
0.07
2.50 0.08
DIP (TV-MAP)
1.8 0.15
2.59
+: 0.10
2.96 0.10
3.11 = 0.12
DIP. (TV-MAP)
1.G3 # 0.09
2.13 = 0.07
2.45
0.08
2.61 #: 0.08
Table 4: PSNR (dB) SSTM of the reconst posterior mean, averaged over 50 KMNIST test images.
a (5%)
#angles:
20
30
n (10%)
#angles:
10
20
30
DIP
21.42/ 0.890
27 0.977
31.11 0.988
32.03
0.991
19.46/ 0.846
24.56/ 0.956
27.27/ 0.974
28.57/
DIP-MCDO
20.95,00.82
28.26/ 0.977
31.
31.65/00994
33.45//0.00 /0.90
MCDO
18.91/0.830
24.76 /0.953
27.72/0.972
29.09 00.99/
Table D shows the test log- likelihood for all the methods and experimental settings under consideration. The peak hignal-to-noise ratio (PSNR) and structural similarity (SSIM) index of posterior mean reconstructions are given in table [ All methods show similar PSNR with the standard DIP (with TV regularisation) obtaining better PSNR in the very ill- ill-posed setting (5 angles) and MCDO obtaining marginally better reconstruction in all others. Despite this, the linearised DIP provides significantly better uncertainty calibration, outperforming all basclines in terms of test log-likel likelihoood in all settings. Figure 3 shows an exemplary character recovered from a simulated observation y (using 20 angles and 5% noise) with both lincarised DIP and DIP-MCDO along with their associated uncertainty maps and calibration plots. DIP-MCDO systematically underestimates
11

---

Published in Transactions on Machine Learning Research (12/2023)
std-dev
In.-DIP
TV-MAP
0.15
0.00
PSNR: 26.35 dB: SSIM: 0.789
0.15
0.00
DIP-MCDO
0.30
0.06
0,08
PSNR: 0.730
0.00
0.00
Figure 5: Reconstruction of 501 501 px² slice of scanned Walnut using lin.-DIP and DIP-MCDO along with their respective uncertainty estimates. The zoomed regions (outlined in red) are given in top-left.
uncertainty for pixels on which the error large, explaining its poor test log-likelihood. The pixel-wise standard deviation provided by linearised DIP (TV-MAP) better correlates with the reconstruction error.
7.1.3 Evaluating the fidelity of sample-based predictive covariance matrix estimation
We evaluate the accuracy of the sampling, conjugate gradient and low rank approximations to the predictive covariance Enly discussed in section S We compute the exact predictive covariance with eq. [3] as reference, which tractable for KMNIST, and do not use patch-based approximations or stabilised covariance ostimators. Table shows that estimating Enjy using samples does not decrease the performance. Using a low-rank approximation J and computing linear solves with PCG lose at most 0.32 nats test log-likelihonk with respect to the exact one, but result in almost an order of magnitude speedup at prediction time.
Table 5: Evaluation our approximate covariance estimation methods in terms of test log- likeelooo 10 KMNIST test images considering the 20 angle (dy = 820) setting and using liP.-DIP (MLL).
Table Test log-likelihodd, PSNR and structural similarity (SSIM) on the Walnut. We compare all lin.-DIP variants with DIP-MCDO.
xx1
xx
10 10
PSNR [3B
SSIM
eq
sampled
sampled (J) eq
(3 PCG)
sumpled Dow
DIP (MLL. J PCG)
Inn-DIP (MLL)
2.00
2.25
2,43
26.35
0.789
2,80 1 0.00
2,80 4 0.00
2.88 4: 0.00
2,62 a 0.09
En. EI. (TV-MAP)
2.21
2.40
2.60
En-DIP (TV-MAP, J PCG)
2.24
2.46
2.65
7.2 Linearised DIP for high-resolution CT
We now demonstrate the approach on real-moasured conc-beam μCT data of a walmut (Der Sarkissian et al. 2019). We reconstruct 501 501 px² slice (d,, 251 001) using a sparse subset. of measurements taken from 60 angles and 128 detector rows (dy 7680), using the U-Net in fig. which has about 3 million parameters. Here, Σxz is too large to store in memory and Eny too expensive to assemble repeatedly, and we use the full suite of approximations in section D Since the Walnut data is not quantised, jitter correction is not needed.
During MLL and Type-ll MAP optimisation, many layers' prior variance goes to o² 2= 0, cf. appendix D This phenomenon known as "automatic relevance dotermination" (Mackay 1996 Tipping 2001). and simplifies our linearised network, preventing uncertainty overestimation. We did not observe this effect when working with KMNIST images and smaller networks. We display the MLL and MAP optimisation profiles for the
12

---

Published in Transactions on Machine Learning Research (12/2023)
marginal std-dev DiP.DIP
marginal sed-dev DCP-MCDO
stó-dev
-DIP TV-MAP
calibration: Q-Q
10
10
(11)
111 ..a8)
se-eer Nt1 2.21]
vus dey
2azt Mide Qol 461
addeee PCG (LA:
cnn quanties
donsity
L00
0-
10
-20
10
0.20
0.25
0.50
0.75
1:30
0,00
0.25
0.50
0.75
1,00
0.00
0.35
0.50
0.75
1.00
-2.5
0.0
2.5
prediiction quantiles
In. - MLLL
Inn.DDIP - TV-MAP
In.DDP
Inn-DIP
TV-MAP J.
DIP-MCDO
Figure 7: The comparison of uncertainty calibration: the pixel-wise error | overlaps with the uncertainties provided by the lin. DIP-MCDO, instcad, severely undercstimates uncertainty. The scale of the pixcl-wise standard deviation (std-dev) obtained including the TV-PredCP matches the absolute error more closely than when the hyperparameters are optimised without. Using J & PCG results in overestimating uncertainty in the tails. LL stands for test log-likelihood.
active layers (i.e. layers with high o3) fig. G Type-II MAP hyperparameters optimisation drives o² to smaller values, compared to MLL. This restricts the linearised DIP prior, and thus the induced posterior, to functions that are smooth in a TV sense, leading to smaller error-bars, ef. fig. 7 As the optimisation of eq. I7 progresses, Eu fall into basins of new minima corresponding to larger lengthscales. This results in more correlated dimensions in the prior, further simplifying the model.
Density estimation described in section 6.3 is con- ducted in double precision (64 bit floating point) since single procision lod to numerical instability in the assembly of Eyg: and also in the estimation of off-diagonal covariance terms for larger patches. In table we report test log-likelihood computed using a Gaussian predictive distribution with covariance patches of sizes x 1, 2 x 2 and 10 x 10 pixels. Mean reconstruction metrics are also reported. Figure 5 displays reconstructed images, uncerlainty maps and calibration plots. In this more challenging task, DIP- MCDO performs poorly relative to the standard DIP formulation eq. (5) in terms of PSNR. DIP-MCDO underest.imates uncertainty, and its uncertainty map is blurred across large sections of the image, placing large uncertainty in well-roconstructed regions and vice-versa. In contrast, the uncertainty map provided by linearised DIP is fine-grained, concentrating on regions of increased reconstruction error. Linearised DIP provides over 2.06 nats per pixel improvement in terms of test log- ligeiiildhood and more calibrated uncertainty estimates, reflected in the Q-Q plot in fig. 0 Furthermore, the use of TV-PredCP prior for MAP optimisation yields a 0.12 nat per pixel improvement over the MLL approach. Interestingly, using low-rank Jacobians and PCG for sampling provides small performance boost when using the TV-PredCP prior. Figure 7 reveals that these approximations result in uncertainty overestimation (a known issue (Antoran et al. 2023 which is compensated by the more restrictive TV-PredCP prior.
4 (In)
in (Up 5)
Las (Out)
102
10
of (In)
o (Up 5)
10
MIL TV-MAP
10
10-2
10
10
10
1020
13000
3000
30000
2003
31032
50481
132001
3000
Itrrsiona
beration
itarationa
Figure 6: Optimisation trajectories for hyperparam- eters of the U-Net's first and last 3 x 3 convolutions (f1, of, lu, of),, last x 1 convolution (fix1,) and noise variance og for the Walnut data.
8 Conclusion
We have proposed a probabilistic formulation of the deep image prior (DIP) that utilises lincarisation of the DIP network around the mode of the loss and Gaussian-linear hierarchical prior on the network
13

---

Published in Transactions on Machine Learning Research (12/2023)
parameters mimicking the total variation prior (constructed via the predictive complexity prior framework). The approach yields well-calibrated uncertainty estimates on tomographic reconstruction tasks based on simulated observations and real-measured μCT data. The empirical results suggest that both the DIP reparametrisation and the TV regulariser provide good inductive biases for high-quality reconstructions and well-calibrated uncertainly estimates. The method is shown to provide by far more calibrated uncertainty estimates than existing MC dropout approaches uncertainty estimation with the DIP. However, this comes at a larger computational cost. Fortunately, since the first appearance of this work, Antoran et al. (2023) have developed techniques that reduce the cost of our linearised DIP inference by two orders of magnitude.
Acknowledgements
The authors would like to thank Shreays Padhy, Marine Schimel, Alexander Teremin, Eric Nalisnick, Erik Daxberger and James Allingham for fruitful discussions. R.B. acknowledges support from the i4health PhD studentship (UK EPSRC EP EE/SS02930/11 and from The Alan Turing Institute (UK EPSRC EP/ AP//510129/// J.L. was funded by the German Research Foundation (DFG; GRK 2224/1) and by the Federal Ministry of Education and Research via the DELETO project (BMBF, project number 05M20LBBB). The work of BJ is partially supported by UK EPSRC grant EP/ GP/VO22259/1 and start-up fund from The Chinese University of Hong Kong. JMHL acknowledges support from Turing AI Fellowship EP/ 4P/VO23756/1 and an EPSRC Prosperity Partnership EP/T000386/
JA acknowledges support from Microsoft. Research, through its PhD Scholarship Programme, and from the EPSRC. This work has been performed using resources provided by the Cambridge Tier-2 system operated by the University of Cambridge Research Computing Service funded by EPSRC Tier-2 capital grant EP 1T022199//
(http:/
References
Jonas Adler, Holger Kohr, and Ozan Öktem. Operator discretization library (ODL.). Softwaree available from https: // gt thub. com/ odlgroup/ odl 2017.
Javier Antorán.
Understanding Uncertainty in Bayesian Neural Networks. Mphil in Machine Learning and Machine Intelligence Thesis, University of Cambridge, 2019.
Javier Antorán, James Allingham, and José Miguel Hernández-Lobato. Depth uncertainty neural networks Advances in Neural Information Processing Systems, 33:10620 10634, 2020.
Javier Antorán, Umang Bhatt, Tameem Adel, Adrian Weller, and José Miguel Hernández-LLoatt Getting a {clue}: A method for explaining uncertainty estimates. In International Conference on Learning Representations, 2021. URL
Javier Antorán, James Allingham, David Janz, Erik Daxberger, Eric Nalisnick, and José Miguel Hernández- Lobato. Linearised Laplace inference in networks with normalisation layers and the neural g-prior. Fourth Symposium on Advances in Approximate Bayesian Inference, AABI 2022, 2022.
Javier Antorán, David Janz, James Urquhart Allingham, Erik A. Daxberger, Riccardo Barbano, Eric T. Nalisnick, and José Miguel Seemándee--obatee Adapting the linearised Laplace model evidence for modern deep learning. In Kamalika Chaudhuri, Stefanie Jegelka, Le Song, Csaba. Szepesvári, Gang Niu, and Sivan Sabato (eds.), International Conference on Machine Learning, volume 162 of Proceedings of Machine Learning Research, pp. 796 821. PMLR, 2022. URL httptttttt/:cceeeeet
mlr. wttppppss/v////ttorrrreeee
html
Javier Anloran, Shreyas Padhy, Riccardo Barbano, Eric Nalisnick, David Janz, and José Miguel Hernández- Lobato. Sampling-bass based inference for large linear models, with application to linearised laplace. In The Eleuenth International Conference on Learning Representations, 2023. URL https:://ppenneviev..e
fformm?d==2oDyXvvsped
Vegard Antun, Francesco Renna, Clarice Poon, Ben Adcock, and Anders C. Hansen. On instabilities of deep learning in image reconstruction and the potential costs of AI. Proc. Nat. Acad. Sci., 117(48))308883300) 2020.
14

---

Published in Transactions on Machine Learning Research (12/2023)
Simon Arridge, Peter Maaß, Ozan Öktem, and Carola-Bibiane Schönlieb. Solving inverse problems using data-driven models. Acla Numer., 28:1-174, 2019.
Arsenii Ashukha, Alexander Lyzhov, Dmitry Molchanov, and Dmitry Vetrov. Pitfalls of in-domain uncertainty estimation and ensembling in deep learning. Preprint, arXiv:2002.06477, 2020.
Daniel Otero Baguer, Johannes Leuschner, and Maximilian Schmidt. Computed tomography reconstruction using deep image prior and learned reconstruction methods. Inverse Problems, 36(9):094004, 2020.
Riccardo
Barbano, Simon Arridge, Bangti Jin, and Ryutaro Tanno. Uncertainty quantification for medical image hests. In Biomedicall Image Synthesis and Simallation: Methods and Applications, pp- 601-641. Elsevier, 2021.
Riccardo Barbano, Javier Antorán, José Miguel Hernández-Lobato and Bangti Jin. A probabilistic deep image prior over image space. In Fourth Symposium Advonces in Approzimate Bayesion Inference, 2022a.
Riccardo Barbano, Johannes Leuschner, Javier Antorán, Bangti Jin, and José Miguel Hernández-L.obato. Bayesian experimental design for computed tomography with the linearised deep image prior. In JCML2082 Workshop on Adaptive Experimental Design and Active Learning in the Real World, 2022b.
Riccardo Barbano, Johannes Leuschner, Maximilian Schmidt, Alexander Denker, Andreas Hauptmann, Peter Maaß, and Bangti Jin. An educated warm start for deep image prior-based micro CT reconstruction. IEEE Trans. Comput. Imag., 8:1210-1222, 1222, 2022c.
Riccardo Barbano, Javier Antorán, Johannes Leuschner, José Miguel Hernández-Lobato, Zeljko Kereta, and Bangti Jin. Fast and painless image reconstruction in deep image prior subspaces. Preprint, arXiv:2302.10279, 2023.
Semih Baruteu, Doãa Gürsoy, and Aggelos K. Katsaggelos. Compressive ptychography using deep image and generative priors. Preprint, ar arXiv:2005.02307 2022.
Umang Bhatt, Javier Antorán, Yunfeng Zhang, Q. Vera Liao, Prasanna Sattigeri, Riccardo Fogliato, Gabrielle Gauthier Melançon, Ranganath Krishnan, Jason Stanley, Omesh Tickoo, Lama Nachman, Rumi Chunara, Madhulika Srikumar, Adrian Weller, and Alice Xiang. Uncertainty as form of transparency: Measuring, communicating, and using uncertainty. In Marion Fourcade, Benjamin Kuipers, Seth Lazar, and Deirdre K. Mulligan (cds.), AIES '21: AAAT/ACM Conference AI, Ethics, and Society, Virtual Event, USA, May 19-21, 2021, pp. 401-413, ACM, 2021. URL https://doi: 111p://t./........//.......
3462571
Charles Blundell, Julien Cornebise, Koray Kavukcuoglu, and Daan Wierstra. Weight uncertainty in neural networks. In Proceedings of the 32nd International Conference on Machine Learning, PMLR 37, pp. 1613-1622, 2015.
Ashish Bora, Ajil Jalal, Eric Price, and Alexandros G. Dimakis. Compressed sensing using generative models. In Doina Precup and Yee Whye Teh (eds.), Proceedings of the 34th International Conference on Machine Learning, volume 70 of Proceedings of Machine Learning Research, pp. 337-544, 2017.
Antonin Chambolle, Vicent Caselles, Daniel Cremers, Matteo Novaga, and Thomas Pock. An introduction to total variation for image analysis. In Theoretical foundations and numerical methods for sparse recovery, pp. 263 340. de Gruyler, 2010.
Zezhou Cheng, Matheus Gadelha, Subhransu Maji, and Daniel Sheldon. A Bayesian perspective the deep image prior. In IEEE Conference on Computer Vision and Pattern Recognition, pp. Vision Foundation IEEE, 2019. URL https://doi.
Computer
21009//VPR..019.00055
Tarin Clanuwat, Mikel Brizar Asanobu Kitamoto, Alex Lamb, Kazuaki Yamamoto, and David Ha. Deep learning for classical Japanese literature. In 32nd Conference on Neural Information Processing Systems (NeurIPS 2018), Workshop Machine Learning for Creativily and Design, 2018.
15

---

Published in Transactions on Machine Learning Research (12/2023)
Jianan Cui, Kuang Gong, Ning Guo, Chenxi Wu, Kyungsang Kim, Huafeng Liu, and Quanzheng Li. Populational and individual information based PET image denoising using conditional unsupervised learning. Phys. Med. & Biol., 66(15):15500, 2021.
Mohammad Zalbagi Darestani and Reinhard Heckel. Accelerated MRT with un-trained neural networks. IEEE Trans. Comput. Imag., 7:724 733, 2021.
Erik Daxberger, Agustimus Kristiadi, Alexander Immer, Runa Eschenhagen, Matthias Bauer, and Philipp Hennig. Laplace redux- effortless Bayesian deep learning. In Marc' Aurelio Ranzato, Alina Beygekzimer, Yann N. Dauphin, Percy Liang, and Jennifer Wortman Vaughan (eds.), Advances in Newral Information Processing Systems 34, pp. 20089-20105 2021a. URL
haa1://22558870007244444600666a2 http://awaaaeeabbaaaaaaaaattttaa html,
Erik Daxberger, Eric Nalisnick, James U. Allingham, Javier Antorán, and Jose Miguel Hennadee--Lobaoo. Bayesian deep learning via subnetwork inference. In Marina Meila and Tong Zhang (eds.), Proceedings of the 38th International Conference on Machine Learning, pp. 2510-2521, 2021b. URL https: https://proceedingg
mlr. httpa////dxeeegeee.aaaaae html
Henri Der Sarkissian, Felix Lucka, Maureen van Eijnatten, Giulia Colacicco, Sophia Bethany Coban, and K. Joost Batenburg. Cone-Beam X-Ray CT Data Collection Designed for Machine Learning: Samples 1-8, 2019. URL https://ddi.org: w20.///e............ 2686726 Zenodo.
Kun Dong, David Eriksson, Hannes Nickisch, David Bindel, and Andrew Gordon Wilson Scalable log determinants for Gaussian process kernel learning. In Isabelle Guyon, Ulrike von Luxburg, Samy Bengio, Hanna M. Wallach, Rob Fergus, S. V. N. Vishwanathan, and Roman Garnett (eds.), Advances in Neural Information Processing Systems 30, pp. 6327 6337, 2017. URL https://ppoceeddiii bibstract.htan
nwurips..c/papen.
Heinz W. Engl, Martin Hanke, and Andreas Neubancr. Regularization of Inverse Problems. Kluwer, Dordrocht, 1996. ISBN 0-7923-4157-0.
Andrew Y. K. Foong, David R. Burt, Yingzhen Li, and Richard E. Turner. On the expressive- ness of approximate inference in Bayesian neural networks. In Hugo Larochelle, Marc' Aurelio Ran- zato, Raia Hadsell, Maria-Florina Balcan, and Hsuan-Tien Lin (eds.), Advances in Newral Infor- mation Processing Systems 33, 2020. URL httts::/prroceeding wewpndiiinnnnnuuiingggree neurips. httppappr/222//haaae/ bBcra41BBbb00003111000700011111 htt.://sc......oo.bb.abbaaaaa.a. html
Vincent Fortuin, Adrià Carriga-Alonso, Florian Wenzel, Gunnar Ratsch, Richard E. Turner, Mark van der Wilk, and Laurence Aitchison. Bayesian neural network priors revisited. In Third Symposium on Advances in Approzimate Bayesian Inference, 2021. URL httpss//openreviee/ httperewwwwieeeeeeeeeeeeeeeeee http://orummdd-xxakkWooooooo
Jacob R. Gardner, Geoff Pleiss, Kilian Q. Weinberger, David Bindel, and Andrew Gordon Wilson. GPyTorch:: Blackbox matrix-matrix Gaussian process inference with GPU acceleration. In Samy Bengio, Hamna M. Wallach, Hugo Larochelle, Kristen Grauman, Nicolò Cesa- Bianchi, and Roman Garnett (eds.), Advances in Neural Information Processing Systems 31, pp. 7587-7597, 2018. URL https::/prooceednggs neuripp abb-a20000aa---saat.t.....a..a.. .html
Adrià Garriga-Aonsso, Laurence Aitchison, and Carl Edward Rasmussen. Deep convolutional networks as shallow Gaussian processes. In International Conference on Learning Representations, 2019. URL
Mark N. Gibbs and David J. C. MacKay. Efficient implementation of Gaussian processes for interpolation. http://www/ httt:///ww.iiferennee phy. cam ac /at..///cccaaa/abbrrrrttss//aa/ htmm, 1996.
Kuang Gong, Ciprian Catana, Jinyi Qi, and Quanzheng Li. PET image reconstruction using deep image prior. IEEE Trans. Med. Imag., 3855)1665511665, 2019.
Peter Guttorp and Tilmann Gneiting. On the Whittle-Matórn correlation family. NRCSE Technical Report No. 80, University of Washington, 01 2005.
16

---

Published in Transactions on Machine Learning Research (12/2023)
Nathan Halko,
Martinsson, and Joel A. Tropp. Finding structure with randomness: Probabilistic
algorithms for constructing approximate matrix decompositions. STAM Rev. 53(2):217 288, 2011. URL
https://doi.org// 1tt.:/...rrg/11111/0000000000000 1137/0007180/
Horace He and Richard Zou. functorch: JAX-like composable function transforms for PyTorch. https:
2021.
Reinhard Heckel and Paul Hand. Deep decoder: Concise image representations from untrained non-
convolutional networks. In ICLR, 2019. URL https://oppmrvvis..ee/////////7p
Tapio Hclin, Nuutti Hyvönen, and Juha-Pekka Puska. Edge-promoting adaptive Bayesian experimental
design for x-ray imaging. STAM J. Sci. Compul. 44(3):B506 B530, 2022. URL https://doi.orgg
1137 4139/0011409330
Emiel Hoogeboom, Taco S. Cohen, and Jakub M. Tomezak. Learning discrete distributions by dequantization. Preprint, 20X1:2001.11235,, 2020.
Jiri Hron, Alex Matthews, and Zoubin Ghahramani. Variational Bayesian dropout: pitfalls and fixes. In Jennifer Dy and Andreas Krause (eds.), Proceedings of the 35th International Conference on Machine Learning, volume 80 of Proceedings of Machine Learning Reseorch, pp. 2019-20000 2020- PMLR, 10-15 Jul 2018.
Alexander Immer, Matthias Bauer, Vincent Fortuin, Gunnar Rätsch, and Mohammad Emtiyaz Khan. Scalable marginal likelihood estimation for model selection in deep learning. In Marina Meila and Tong Zhang (eds.), Proceedings of the 38th International Conference on Machine Learnóng, volume 139 of Proceedingg of Machine Learning Research, pp. 4563-4573, PMLR, 2021a. URL http::/procceddinn mlr press/v139/ 1mmer21a.htm.
Alexander Immer, Maciej Korzepa, and Matthias Bauer. Improving predictions of Bayesian neural nets via local linearization. In Arindam Banerjee and Kenji Fukumizu (eds.), The 24th International Conference on Artificial Intelligence and Statistics, volume 130 of Proceedings of Machine Learning Research, pp. 703-711. PMLR, 2021b. URL http::/ppocceddii
mlr gress/v130/ htt3300mmmm22a....mmm hmmer21a.html html
Kazufumi Ito and Bangti Jin. Inverse Problems: Tikhonov Theory and Algorithms, volume 22. World Scientific, 2014.
Arthur Jacot, Clément Hongler, and Franck Gabriel. Neural tangent kernel: Convergence and gener- alization in neural networks. In Samy Bengio, Hanna M. Wallach, Hugo Larochelle, Kristen Grau- man, Nicolò Cesa-Bianchi, and Roman Garnett (eds.), Advances in Neural Information Process- ing Systems 31, pp. 8580-8589, 2018. URL https::/proceednngs.oo wwt.:nneeerrccppppeeeeeeeee 1a2eeafff33 Abstrrctt.abttttt..t....m html
Jari Kaipio and Erkki Somersalo. Statistical and Computational Inverse Problems. Springer- Verlag, New York, 2005. ISBN 0-387-22073-9.
Mohammad Emtiyaz Khan, Alexander Immer, Ehsan Abedi, and Maciej Korzepa. Approximate inference turns deep networks into Gaussian processes. In Hanna M. Wallach, Hugo Larochelle, Alina Beygelzimer, Florence d' d'Alch6- Buc, Emily B. Fox, and Roman Garnett (ods.), Advances in Neural Information Processing Systems 32, pp. 3088-3098, 2019.
Tobias Knopp and Micro Grosser. Warmstart approach for accelerating deep image prior reconstruction in dynamic tomography. Proceedings of Machine Learning Research, Medical Imaging with Deep Learning 2022, 13 pp., 2022.
Benjamin Kompa, Jasper Snoek, and Andrew L. Beam. Second opinion needed: communicating uncertainty in medical machine learning. NPJ Digital Medicine, 4(1):1 6, 2021.
Max-Heinrich Laves, Malte Tölle, and Tobias Ortmaier. Uncertainty estimation medical image denoising with Bayesian deep image prior. In Uncertainty for Safe Utilization of Machine Learning in Medical Imaging, and Graphs in Biomedical Image Analysis, pp. 81-96. Springer, 2020.
17

---

Published in Transactions on Machine Learning Research (12/2023)
Jachoon Lee, Lechao Xiao, Samuel S. Schoenholz, Yasaman Bahri, Roman Novak, Jascha Sohl-Dickstein, and Jeffrey Pennington. Wide neural networks of any depth evolve as linear models under gradient descent. In Hanna M. Wallach, Hugo Larochelle, Alina Beygelzimer, Florence d' A'Achéé-Buc, Emily B. Fox, and Roman Garnett (eds.), Advances in Newral Information Processing Systems 32, pp.
2019. URL https: html
F. C. Leone, L. S. Nelson, and R. B. Nottingham. The folded normal distribution. Technomelrics, 3:543-555 1961. ISSN 0040-1706. URL httpa///ww...at.////////aaaaaaan
Jiaming Liu, Yu Sun, Xiaojian Xu, and Ulugbek S Kamilov. Image restoration using total variation regularized doep image prior. In ICASSP 2019, 2019. URL httpss///ol..og//00.110///...0. 20098888868
David J. C. Mackay. Bayesian non-lincar modeling for prediction competition. In Mazimum Entropy and Bayesian Methods, pp. 221-222222 211, 1996.
David John Cameron Mackay. Bayesian Melhods for Adoplive Models. PhD thesis, California. Institute of Technology, California, USA, 1992.
Wesley Maddox, Shuai Tang, Pablo Garcia Moreno, Andrew Gordon Wilson, and Andreas C. Damianou. Fast adaptation with lincarized neural networks. In Arindam Banerjec and Kenji Fukumizu (eds.), The 24th International Conference on Artificial Intelligence and Statistics, volume 130 of Proceedings of Machine Learning Research, pp. 2737-2745, PMLR, 2021. URL http://proceedingg.uu mlr ppesss//1300mmmddx22211 http://iiiaa.a...oooooo/////////
html
Wesley J. Maddox, Pavel Izmailov, Timur Garipov, Dmitry P. Vetrov, and Andrew Gordon Wilson. A simple baseline for Bayesian uncertainty in deep learning. Advances in Neural Information Processing Systems, 32, 2019.
Per-Gunnar Martinsson and Joel A. Tropp. Randomized numerical linear algebra: Foundations and algorithms. Acta Numer., 29:403 572, 2020.
Kenneth O. McGraw and S. P. Wong, The descriptive use of absolute differences between pairs of scores with a common mean and variance. J. Educat. Stat., 19(2): 1994.
Eric T. Nalisnick, Jonathan Gordon, and José Miguel Hernández-Lobate Predictive complexity priors. In Arindam Banerjee and Kenji Fukumizu (eds.). The 24th International Conference on Artificial Intelligence and Statistics, volume 130 of Proceedings of Machine Learning Research, pp. 694-702. PMLR, 2021. URL html
http://proceedding mlr http://ssmwm...eesssaemem Pttess/v130/naaiinn
Roman Novak, Lechao Xiao, Yasaman Bahri, Jaehoon Lee, Greg Yang, Jiri Hron, Daniel A. Abolafia, Jeffrey Pennington, and Jascha Sohl-Dickstein. Bayesian deep convolutional networks with many channels are Gaussian processes. In 7th International Conference on Learning Representations. OpenReview.nee 2019. URL https:://openrevieer.e wtt.a//eeeeeeeeeeee.o.oooodoeaad
Roman Novak, Lechao Xiao, Jiri Hron, Jaehoon Lee, Alexander A. Alemi, Jascha Sohl-Dickstein, and Samuel S. Schoenholz. Neural tangents: Fast and easy infinite neural networks in Python. In 8th International Conference on Learning Representations, 2020. URL
Gregory Ongic, Ajil Jalal, Richard G. Baraniuk, Christopher A. Metzler, Alexandros G. Dimakis, and Rebecca Willett. Deep learning techniques for inverse problems in imaging. IEEE J. Sel. Areas Inform. Theory, pp- 39 56, 2020.
Carl Edward Rasmussen and Christopher K. I. Williams. Goussian Processes for Machine Learning. The MIT Press, 2005. ISBN 026218253X.
Olaf Ronneberger, Philipp Fischer, and Thomas Brox U-net: Convolutional networks for biomedical image segmentation. In International Conference on Medical Image Computing and Computer-Assisteee Intervention, pp. 234-241, 2015.
18

---

Published in Transactions on Machine Learning Research (12/2023)
Leonid I. Rudin, Stanley Osher, and Emad Fatemi. Nonlinear total variation based noise removal algorithms.
Physica D, 60(1-4):259 268, 1992.
Walter Rudin. Fourter Analysis Groups. John-Wiley, Wiley, New York-London, 1990.
Matthias W. Seeger and Hannes Nickisch. Large scale Bayesian inference and experimental design for sparse linear models. SIAM J. Imaging Sci., 4(1):166 199, 2011. URL /tttpp///bol..rrr/11113////0///0
Bernard W. Silverman. Density Estimation for Statistics and Data Analysis. Chapman Hall, London, 1986.
ISBN 0-412-24620-1.
Jasper Snoek, Yaniv Ovadia, Emily Fertig, Balaji Lakshminarayanan, Sebastian Nowozin, D. Sculley, Joshua V. Dillon, Jie Ren, and Zachary Nado. Can you trust your model's uncertainty? evaluating predictive uncertainty under dataset shift. In Hanna M. Wallach, Hugo Larochelle, Alina Beygelzimer, Florence d' d'Alché-Buc, Emily B. Fox, and Roman Garnett (eds.), Advances in Neural Information Pro- cessing Systems 32, pp. 13969 13980, 2019. URL httpe://proceedingss nwwrips.eerppp... hew.rrpp..cc/ppppe///////aaaaaaa Absr:a7 html
Andrew M. Stuart. Inverse problems: Bayesian perspective. Acla Numer. 19:451 559, 2010. ISSN 0962-4929. URL
Andrey N. Tikhonov and Vasiliy Y. Arsenin. Solutions of Ill-posed Problems. John Wiley & Sons, New York-Toronio Ont.-London, 1977.
Michael E. Tipping. Sparse Bayesian learning and the relevance vector machine. J. Mach. Learn. Res., pp- 211 244, 2001.
Malte Tölle, Max-Heinrich Laves, and Alexander Schlaefer. A mean-field variational inference approach to deep image prior for inverse problems in medical imaging- In Mattias P. Heinrich, Qi Dou, Marleen de Bruijue, Jan Lellmann, Alexander Schlaefer, and Floris Erust (eds.), Medical Imaging with Deep Learning, 7-9 July 2021, Lübeck, Germany, volume 143 of Proceedings of Machine Learning Research, pp. 745-760. PMLR, 2021. URL https://prrceedding mlr wwt.mm..r...........m. ht14443/tole22a.aata html
Dmitry Ulyanov, Andrea Vedaldi, and Victor Lempitsky. Deep image prior. In Proceedings of the IEEE Conference on Computer Vision and Pottern Recognition (CVPR), pp. 9446-9454, 2018.
Dmilry Ulyanov, Andrea Vedaldi, and Victor Lempilsky. Deep image prior. Int. J. Compul. Vis., 128(7): 1887-1888, 2020. ISSN 1573-1405. URL http:///aa..rrgg/..000////11113/
Wim van Aarle, Willem Jan Palenstijn, Jan De Beenhouwer, Thomas Altantzis, Sara Bals, K. Joost Batenburg, and Jan Sijbers. The ASTRA Toolbox: A platform for advanced algorithm development in electron tomography. Ultramicroscopy, 157:35 47, 2015. URL http::///ooi.rgg/....1 1tt.//......timiiccccc... ultramic
2015.05.0002
Francisca Vasconcelos, Bobby He, Nalini Singh, and Yee Whye Tch. UncertaINR: Uncertainty quantification of end-to-end implicit neural representations for computed tomography. Preprint, arXiv:2202.10847 2022.
Ge Wang, Jong Chul Ye, and Bruno De Man. Deep learning for tomographic image reconstruction. Nature Mach. Intell., ((12):737-748, 2020.
Hengkang Wang, Taihui Li, Zhong Zhuang, Tiancong Chen, Hengyue Liang, and Ju Sun. Early stopping for deep image prior. Preprint,
2021.
Max Welling and Yee Whye Teh. Bayesian learning via stochastic gradient Langevin dynamics. In Lisc Getoor and Tobias Scheffer (eds.), Proceedings of the 28th International Conference on Machine Learning, pp. 681-688880 688. Omnipress, 2011. URL https::/icmlccc///// wttpccc/c//cpcc//////// 1e98_icmlpaperrree.m
James T. Wilson, Viacheslav Borovitskiy, Alexander Terenin, Peter Mostowsky, and Mare Peter Deisenroth. Pathwise conditioning of Gaussian processes. J. Mach. Learn. Res., 22:105:1-105:47, 2021. URL http: /twpm//ogm//aaaaeeeeee org/paapers/v2//20-1888
19


Reviews:
[{"review_id": "xrOEl9nWLm", "reviewer": "Reviewer_M8N9", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "**Strengths:**\n\n- The proposed algorithm, as well as the presented computational enhancements, are simple and straightforward.\nThe model is adequately evaluated in a series of empirical experiments.\n- According to the provided experiments, the calibration of the uncertainty estimates and the quality of the reconstructed images are both qualitatively and quantitatively competitive.\n\n**Weaknesses:**\n\n- Discussion of the limitations of the work. The authors note that the uncertainty estimate provided in this paper can only be obtained with respect to a locally linear approximation of the DIP model around its optimized parameters. Is this an adequate approximation? Moreover, the algorithm appears to be quite expensive in terms of wall-clock time (Table 1). Is this a unique weakness of the presented approach? How does it compare with competing methods?\n- Limited empirical comparisons. Is there a reason why the paper only compares against a few of the papers discussed in the related works section (Appendix A)? Moreover, is it appropriate to leave the related works section in the Appendix?", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "Yes", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "The authors present a novel method for uncertainty prediction in deep image prior (DIP) -based inverse imaging. Uncertainty estimates are provided by a Bayesian formulation of the inverse imaging problem combined with a linearized DIP model. The authors further improve the algorithm by proposing various approximations that greatly reduce computation time while incurring negligible performance reductions.", "claims_and_evidence_yn": "Yes", "claims_explanation": "", "audience_yn": "Yes", "audience_explanation": "", "requested_changes": "*Inclusion of a related works section in the main work*: At present, the manuscript is missing a related works section. This is deferred to the appendix, presumably due to space restrictions. I find this solution somewhat unsatisfactory, since clearly delineating the novelty of the algorithm and placing it in the context of its surrounding literature is a crucial step to understanding a research paper. I do emphasize that this is a more minor concern -- perhaps the authors can at least mention that the related works are in the appendix?\n\n*Discussion of limitations*: It may also be fruitful to describe the limitations of the present work. For example, how does the computational cost of the locally linear approximations related to other DIP-based inverse imaging algorithms? How restrictive is the Laplace approximation?\n\nMinor:\n\"The discrepancy among plausible reconstructions in the posterior indicates uncertainty.\" <- This sentence is unclear to me. Are the authors just stating that $p(x|y)$ models the uncertainty of the reconstruction $x$ given the provided Bayesian model?", "broader_impact_concerns": "While a broader impact statement is certainly welcome, I do not see any clear ethical concerns with the presented work that necessitates the addition of one.", "additional_comments": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "wpdzjfJUiP", "reviewer": "Reviewer_955y", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "The benefit of such a formulation is the characterisation of uncertainty with respect to the denoised CT images, which can be of particular usefulness in the case of limited-angle measurement.\n\nThe disadvantage is that the formulation is highly computationally expensive and the practicality of the proposed methodology is put into doubt.", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "Yes", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "This submission suggests an application of deep image prior (DIP) to be integrated with total variation regularisation for computed tomography. Authors add a post-processing step to the DIP-based CT image denoising in the framework of maximum a posteriori (MAP), with the initialised iteration point of the MAP optimisation problem as the output of a deep image prior and with other regularisation terms derived from total variation.\n\nTechnically, there is no novelty in the DIP design itself. The methodology innovation in this manuscript is the post-processing MAP step.", "claims_and_evidence_yn": "Yes", "claims_explanation": "", "audience_yn": "Yes", "audience_explanation": "", "requested_changes": "My specific questions on the manuscript:\n\n(1) The DIP has a technicality that when to stop training is unapparent to decide. With such a complicated objective to optimise (equation 17 on page 7), deciding the early stopping seems a challenge. I do not see particular discussion made in the manuscript about the control of early stopping. It will be helpful for the audience to know authors' technique to manipulate the training setting.\n\n(2) The computation of $\\Sigma_{yy}^{-1}$ seems computationally insurmountable. It is unclear to me how authors manage to pull out the MAP optimisation process in the CT setting. The coordination between the updates made to parameters $\\sigma$ and $\\ell$ should be clearly explained. \n\n(3) What are y-axis labels in figure 6?\n\n(4) Why values are missing in table 6 for the latter three methods?\n\n(5) Figure 7 seems to show that the DIP-MCDO has the deviation statistics regarding pixels more concentrated than linearised DIP methods. Is this not a desired property to have a reduced uncertainty? \n\n(6) For sanity check, authors should clearly demonstrate the comparison between the results from \"training the DIP to an optimal TV-regularised setting $\\widehat{x}$\" and that from he propose linearised DIP method.\n\nMinor questions:\n\n- What is \"nats'?\n\n- How is 'task-agnostic pretraining' set up in section 2.3?\n\n- Define $J_d$ in equation 20.\n", "broader_impact_concerns": "No ethical implications or concerns noticed.", "additional_comments": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "GoWjVfF7V7", "reviewer": "Reviewer_hYVv", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "Strengths\n\n+Studies important problem\n\n+Outperforms state-of-the-art DIP uncertainty estimation methods\n\n+Generally well-written\n\n\nWeaknesses\n\n-Provides only an uncertainty estimate around a mode. If posterior is multimodal this won't be accurately reflected in the results. Paper convincingly argues this \"shortcoming\" improves interpretability as practitioners may not benefit from multiple modes anyway.\n\n-Section 2 places a Gaussian prior on theta (and implicitly x), but this doesn't show in p(x) in equation (7). Is equation (7) more accurately described as p(x|\\theta), with p(x)=p(x|\\theta)*p(\\theta)?\n\n-Comparisons are restricted to DIP and data is toy-like (Walnut has simpler structure than typical CT scenes).\n\n\n\nTypos:\n\nPg 4: unormalized-->unnormalized\n\nFigures 4 and 5: One of the images seems to be missing a colormap and it's unclear if \"TV-MAP\" belongs.\n", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "Yes", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "Develops a technique to estimate the uncertainty associated with CT reconstructions formed by Deep Image Prior with TV regularization. The proposed technique works by linearizing (first order Taylor expansion) DIP around its optimized parameterized, which can be used to generate error bars for the reconstructions. The proposed method compares favorably to existing DIP uncertainty estimation techniques.\n", "claims_and_evidence_yn": "Yes", "claims_explanation": "", "audience_yn": "Yes", "audience_explanation": "", "requested_changes": "If the authors could answer my question about p(x) vs p(x|\\theta), I'm happy with the paper as is. While the paper's focus is relatively niche (uncertainty estimation for DIP), it makes a novel and well-validated contribution to that niche.\n", "broader_impact_concerns": "None", "additional_comments": "", "extra_scores": {}, "extra_text": {}}]

Author rebuttal:

Meta-review: The paper is mostly good to go. 
I only have the following requirement: Please add a formal description of the final algorithm proposed, taking into account the different steps. This can be in the main text or in the appendix.
Decision: Accept (poster)