UncertaINR: Uncertainty Quantification of End-to-End Implicit Neural Representations for Computed Tomography
Authors: Francisca Vasconcelos
Bobby He
Nalini M Singh
Yee Whye Teh
Venue: Accepted by TMLR ()

Abstract: Implicit neural representations (INRs) have achieved impressive results for scene reconstruction and computer graphics, where their performance has primarily been assessed on reconstruction accuracy. As INRs make their way into other domains, where model predictions inform high-stakes decision-making, uncertainty quantification of INR inference is becoming critical. To that end, we study a Bayesian reformulation of INRs, UncertaINR, in the context of computed tomography, and evaluate several Bayesian deep learning implementations in terms of accuracy and calibration.  We find that they achieve well-calibrated uncertainty, while retaining accuracy competitive with other classical, INR-based, and CNN-based reconstruction techniques. Contrary to common intuition in the Bayesian deep learning literature, we find that INRs obtain the best calibration with computationally efficient Monte Carlo dropout, outperforming Hamiltonian Monte Carlo and deep ensembles. Moreover, in contrast to the best-performing prior approaches, UncertaINR does not require a large training dataset, but only a handful of validation images.

Published in Transactions on Machine Learning Research (04/2023)
UncertalNR: Uncertainty Quantification of End-to-End Implicit Neural Representations for Computed Tomography
Francisca Vasconcelos"
fruncisca@beeraeey..cd edu
Department of Electrionl Engineering and Computer Science University of California, Berkeley
Bobby He
hobby.he@staass.oo
Department of Stalistics University of Ouford
Nalini Singh
amsiingh &eiitt edu
Computer Science and Artificial Intelligence Laboratory Massachusetts Institute of Technology
Yee Whye Teh
w.w.ch@statssoo or.
Department of Statistics University of Ouford
Reviewed on OpenReview: https: // net/ forum? id- ydMMBgYufX
Abstract
Implicit neural representations (INRs) have achieved impressive results for scene reconstruc- lion and computer graphics, where their performance has primarily been assessed on recon- struction accuracy. As INRs make their way into other domains, where model predictions inform high-stakes decision-makinn critical. To that end, we study a Bayesian reformulation of INRs, UncertaINR, in the con- text of computed tomography, and evaluate several Bayesian deep learning implementations in terms of accuracy and calibration. We find that they achieve well-calibrated uncertainty, while retaining accuracy competitive with other classical, INR-based, and CNN-based recon- struction techniques. Contrary common intuition the Bayesian deep learning literature, find that INRs obtaín the best calibration with computationally efficient Monte Carlo dropout, outperforming Hamiltonian Monte Carlo and deep ensembles. Moreover, in con- trast to the best- west-performing prior approaches, UncertaINR does not require a large training dataset, but only handful of validation images.
uncertainty quantification of INR inference is becoming
1
Introduction
Implicit neural representations (TNRs) are recent technique for capturing complex, coordinate-basety signals.
achieving impressive results in novel view synthesis (Mildenhall ct al. 2020
Niemeyer et al. 2020;
Saito et al.
2019
Sitzmann el al., 2019), shape representation (Chen & Zhang 2019
Deng el. al. 2020 Genova et al.
2019 2020 Jiang et al.) 2020 Park et al.) 2019), and texture synthesis (Henzler et al. 2020 Oechsle et al,
2019
the context of image reconstruction, INRs represent images as continuous functions mapping coordinates pixel values, (x, (0, 1]. Typically ʃ is parameterized by small neural network (NN) and trained with gradient-based optimization, given observations S. This continuous INR formulation offers several benefits to discrete array representations, e.g. enabling signal processing at arbitrary resolutions (thereby avoiding the so-called "curse of discretization" (Mescheder 2020 and improved memory efficiency
Dupont et al. 2021
Equal contribution.
This work was done while author was the University of Oxford.

---

Published in Transactions on Machine Learning Research (04/2023)
Noiey CT Scan Megurenest Dats
UncertaINR
X-Ray Detertor
Sinegram
Poal Cardinatos
5p0r)
f(
X-Ray Source
Recesstreccal Image
Contrage
Culbration
(a) UncertalNR Framework
(b) Computed Tomography Setup
Figure 1: (a) From a noisy CT scan, UncertaLNR reconstructs a high-quality image of the 2D imaging subject cross-section. Further, UncertaINR achieves well-calibrated UQ over the reconstructed image, enabling doctors use the predicted variance determine regions where the reconstruction may have failed. (b) A CT scan of attemmation-coefffcient image (" y) results in sinogram (r)
Despite these appealing properties, existing INR. applications have primarily focused their scope on either: 1) predictive representation accuracy 2) visual plausibility of predictions extrapolated from given training signal S, However, there are settings e.g. Magnetic Resonance Imaging (MRI) and Computed Tomography (CT) Tannii et al. 2020 in which INRs have proven promising and for which uncertainty quantification (UQ) is highly desirable. In such applications, only an underdetermined set of observations S is available, which is exacerbated by the high-dimensional, non-convex nature of NN parameter landscapes. As a result, many different INR parameter values may fit the observed data equally well, but yield vastly different predictions (some of which may be poor, e.g. Figure of [Taniik al. (2021)) when extrapolating unobserved regions of coordinate space. Hence, in such scenarios, calibrated predictive uncertainty is crucial alongside high reconstruction accuracy.
This work assesses the UQ capabilities of INRs in the applied setting of medical imaging, specifically com- puted tomography (CT). This setting is apt for INR UQ due to its high-stakes nature even small image artifact could result in misdiagnosis and underdetermination of CT sinogram observations. Morcover, UQ could be leveraged to reduce healthcare costs via automated triage, e.g. by assigning images with varying degrees of uncertainty healthcare providers of relevant expertise. Furthermore, each CT scan measurement exposes the patient to harmful radiation meaning calibrated model uncertainty could enable techniques, such active learning (Cohn et al. 1996), to inform more efficient measurement collection and reduce neg- ative radiation exposure effects. Although uncertainty has been quantified in deep-learning based medical Laves et al. 2022). medical datasets are often small and apparatus-specife
imaging (Barbano et al.) 2021
posing significant gencralization
challenges for large data-driven approaches (Abramoff et al. 2018 De Fauw
et al. 2018 Hosny al.
2018 Yasaka & Abe 2018 Zhang al. 2020 Meanwhile, INRs require little
no training data and have previously demoustrated decent C'T reconstruction accuracy (Tancik et al. 2020
which we now complement with well-calibrated UQ
In light of the aforementioned need for well-calibrated INRs, we propose UncertaINR (Figure [a): l)): Bayesian reformulation of basob image reconstruction. We demonstrate effective application of Bayesian deep- learning (BDL) principles to INRs, in the underdetermined CT reconstruction context, and study the perfor- mance of various established BDL approximate inference approaches: boyyes-b-backprry (BBB) (Blundell ct al. 2015), Monte Carlo dropout (MCD) (Gal & Ghahramani] 2016),, and deep ensembles (DEs) (Laksh- minarayanan al.] 2017). as well as the "gold-standard' but computationally intensive Hamiltonian Monte
Carlo (HMC) (Neal al. 2011 We find that the simple, yet efficient MCD procedure is highly effective
An estimated 29,000 current future cancer cases linked CT scans performed in the United States of America in 2007 alone (De González al. 2009
2

---

Published in Transactions on Machine Learning Research (04/2023)
achieving calibrated INR. UQ. MCD outperforms deep ensembles, countering findings in other BDL setups, like image classification where ensembling is considered state-of-the-art NN UQ approach (Ovadia et al. 2019). MCD also rivals (sometimes outperforming) HMC, which is far more computationally demanding and difficult to hypertune. Ensembling INRs with MCD achieves further performance gains. Overall, Un- certaINR attains well-calibrated uncertainly estimates without sacrificing reconstruction quality relative other classical, INR-basod, and CNN-based reconstruction techniques on realistic, noisy, and underdeter- mined data.
2 Problem Setting and Related Works
2.1 CT Image Reconstruction
We are interested in reconstructing 2D images, represented as functions f(x,8) y) tissue attenuation coef- ficients pixel locations (z,y) within an imaging subject cross-section. As visualized in Figure Db. CT scanners rotate X-ray emitters and detectors around the subject, collecting measurements of the Radon transform (r) of the desired image f rather than actual image values f(r,y),
/ ((x,0) ö(x cos + ysin Φ - r) drdy.
S -
(1)
for view-angles & and X-ray detector radii r. Sa(r) is also known as a sinogram and is not directly human- interpretable. The reconstruction (xxy) from (r) thus constitutes an inverse problem, governed by the inverse Radon transform. Appendix [ further describes the CT measurement physics and image re- construction problem. While the Projection Slice Theorem (Bracewell, 1956) ensures that f can be fully reconstructed from the complete sinogram, practical measurement, data is finite and noisy, making the image reconstruction problem underdetermined and reconstruction UQ desireable.
We assume that image f(x, y) is reconstructed on a finite set of pixels X' x y (assumed to be grid-spaced for comparison of INRs with classical grid-based reconstruction techniques), while sinogram measurements are observed on a finite set of view-angles & and X-ray detector radii R. For notational convenience, we denote the ith sinogram measurement S; for E {1,..., 1 x R}} and jth pixel value Jj for j E (1,..., In a slight abuse of notation, we also denote resulting vectors of pixel values and sinogram measurements f and S, respectively. Thus, the discretization of Eq. is
......,1.. |X x )/).
|xxy| Σ
AgJ = St, Vi; equivalently, Af= = S,
(2)
where the ijth entry of the discretized Radon transform matrix, A, represents pixel j's contribution to the ith prediction measurement, c.g. Aij is zero when pixel j is not along the ray mcasured by S,.
2.1.1 Classical CT Reconstruction Methods
Among classical approaches for CT reconstruction, filtered backprojection (FBP) is one of the simplest ana- lytical reconstruction techniques, providing a simple closed-form estimate of the image. By upweighting high frequencies, FBP enables reconstruction of detailed image structures, but can also emphasize high-frequency noise, resulting in poor image quality. Tterative reconstruction techniques - e.g. the algebraic reconst technique (ART) (Gordon et al., 1970), simultaneous iterative reconstruction technique (SIRT) (Gilbert) 1972), simultancous algebraic reconstruction technique (SART) (Andersen & Kak 1984). and conjugate gradient for least. squares (CGLS) (uaan & Jusem 1996 can miligate these noise ellects.
Another algorithm class frames the reconstruction problem as minimization of a regularized objective,
min || 2 + XT()),
mjju
(3)
3

---

Published in Transactions on Machine Learning Research (04/2023)
where T(f) is regularizer encoding prior on f, with regularization strength X. A regularizer typically used in medical imaging is the total-variation (TV),
Σ V(r(o + 11,)) f(x,u))² + (f(a, - (z, Z.BEXXY
TTv(1)=
(4)
which removes unwanted image noise and artifacts, while preserving important details such as edges (Rudin et al. 1992). The minimization problem in Eq. with TV regularization is solvable via proximal gradient- based techniques, e-g. the fast iterative shrinkage-turesholding algorithm (FISTA-TV) (Beck & Teboulle, 2009). Alternatively, expectation--mxxiiaaiiie (FM) iteratively maximizes the log likelihood of the pro- jections given the estimated image (Dong) 2007). While these regularized methods typically outperform analytic mothods, iterative solutions may be computationally slow and their reconstruction quality can be further improved. Appendix B provides detailed descriptions of these classical techniques.
2.1.2 Deep-Learning CT Reconstruction Methods
Challenges for classical methods have motivated significant recent interest in NNs trained with large-scale datasets to reconstruct high quality CT images from low-dose acquisitions. Because reconstruction requires only a single NN forward pass, rather than a large number of optimization updates, these methods are faster than purely iterative methods. Some doep-lcarning (DL) approaches input analytic low-dose recon- structed CT images into an NN trained to directly produce artifact-free reconstructions from higher-dose Yang et al.] 2018). Alternatively, "unrolled" network architectures (Adler & Oktem 2018 Jin et al. 2017 Wu et al. 2019) solve Eq. 3 by chaining together NN layers such that each layer computes one optimization update. In this work, we compare our methods to the FBP-Unet (Jin et al.) 2017). We also compare to GM-RED (Sun el al. 2021), which uses a deep denoiser trained on the acquired dataset to define a reconstructed image prior lying on a manifold of natural images. We note that while FBP- and GM-RED require large training datasets, our mothod leveraging INRs requires only few validation images for hyperparameter luning.
acquisitions (Chen et al.) 2017a b. Liu & Zhang 2018
Although DL has enabled advances in CT reconstruction (Lell & Kachelrieß 2020), progress has been ham- pered by the specific, expensive, and small nature of CT datascts. This follows from the existence of varions CT-imaging modalities (e.g. helical, spiral, electron beam, and perfusion imaging) and need for individual calibration of CT-imaging apparatuses, which compromises the transferability of DL models. CT data collec- tion also expensive, requiring long hours on costly machines (exposing patients to harmful radiation) and labels by expert practitioners (making large-scale annotation virtually impossible). In result, CT datasets are small relative to those of other areas, such ImageNet (Deng et al., 2009), whose size has proven crucial in enabling state-of-the-art DL performance. In light of these challenges and the aforementioned radiation risk, there is clear need for NN methods that achieve high-quality CT image reconstruction from small dalasels consisting of few measurements. Combined with our desire for calibrated UQ, this motivates the study of INRs in our UncertaINR, framework.
2.2 Implicit Neural Representations
Implicit neural representations (LNRs), implemented via NNs, are functions f-(() mapping coordinates æ to a coordinate-wite feature fo(x) of interest, with parameters trained to match some observed signal S. Due to their general and scalable formulation, INRs have been applied to a wide range of data modalities including: 3D scenes (Sitzmann et al. 2019),, voxel grids (Dupont et al., 2022a; Meschoder et al.) 2019), video (Li et al., 2021), and audio (Sitzmann et al. 2020). ). In this work, we focus on UQ of INRs for CT reconstruction of (z. y) F² and observed sinogram S. We leave exploration of more
2D image f, with pixel inputs complex data. modalities, such as 3D or 4D CT reconstruction, for future work.
In recent years, INRs have grown popular both in computer graphics and unrelated fields like medical imaging (Tancik et al. 2020), arguably inspired by state-of-the-an novel view syuthesis results achieved by neural radiance fields (NeRF) MMildenhall al. 2020).. Since NcRF, INRs have been successfully applied to:
high-resolution 3D scenes from unstructured collections of 2D images (Martin-Brusee et al. (202]): scalable
large scene view synthesis (Tancik et al. 2022; generative modelling (Dupont et al. 2022b
meta-learning
4

---

Published in Transactions on Machine Learning Research (04/2023)
Tancik et al. 2021); ); and image segmentation of medical scans (Khan & Fang 2022 Meanwhile, several
improvements in INR architectures have accompanied these empirical gains, such as random Fourier feature (RFF) (Rahimi & Recht, 2007) encodings {Tancik et al.) 2020) and periodic activation functions (itzmann ct al. 2020), both of which have a tunable frequency hyperparamcter Qg that enables INRs to represent high Irequency functions. In UncertaINR; we adopted RFF encodings, detailed in Appendices F.3 and F.4
and similarly found Ωg critical for decent reconstruction accuracy (Appendix F.5.1 One key challenge for INRs their long evaluation times. However, recent literature has focused on addressing this challenge, successfully leveraging sparse voxel models (Fridovich- Keil et al. 2022) and multiresolution hash-encodings (Müleer et al. 2022 to reduce evaluation times by orders of magnitude.
Recent work has also demonstrated the applicability of INRs to CT image reconstruction. Reed et al. (2021) utilize parametric motion field warped INRs to perform limited view 4D-CT reconstruction of rapidly deforming scenes. (Sun et al. (2021) propose Coordinate-basse
Internal Learning (CoTI.) and Zang et al. (2021) propose IntraTomo, which use INRs to boost the performance of classical reconstruction algorithms, such those discussed in Section 2.1 In ColT., an INR. learns a functional form of the sinogram, receiving sinogram location r),r) as input and outputting projection measurement Sr(r). This functional sinogram generates artificial measurements from view angles not included in the original measurement sinogram. The reconstruction algorithm leverages this artificially INR-enlarged measurement set to achieve improved performance reconstructing image f, over the same algorithm trained on the original, smaller measurement dataset.
We note that no existing INR works have addressed the aforementioned need for calibrated UQ, motivating our proposed UncertaINR framework.
3 UncertalNR: Uncertainty Quantification of INRs for CT
To quantify the uncertainty in reconstructing image f, given sinogram measurments S, we reformulate the CT reconstruction problem, Eq. 3 as one of Bayesian inference. We assune a Gaussian measurement model,
S I ind. N(A.S,o²) Vi (1,...,/$xX11
where o² an assumed known observation noise, and A, is the ith row of the discretized Radon transform A Once prior distribution is placed over f, the posterior distribution over can be computed. The posterior distribution captures both plausible reconstructions of (e.g. via the posterior mean), as well the uncertainty over reconstructions (e.g. via the posterior standard deviation).
An important practical consideration is to choose how the image f is parameterized. In the following we compare two alternative parameterizations: a classical grid-based baseline and our INR-based proposal. For each these parameterizations we will also discuss appropriate priors for f.
Grid-of-Pixels Baseline As baseline, parameterize the image f using a pixel-wise grid representation: specifically, we lake / R|x*> A sensible prior, which prefers smoothness in images while allowing for important details such as edges, is p(f) x exp(-XTrv(f)), where Tfv(f) is the votal-variation of f. In this case the posterior distribution is
4
(8,-A.) (.f-A,))- - 1i--A))-XTivvv) ALwAN)
p([S) exp
Σ
(6)
2n
This is Bayesian extension to the grid-based methods of Section 2.1.1 with Eq. 3 corresponding to the maximum a posteriori (MAP) solution to Eq. 6 In our experiments we compare INR-based inferences to this discretized baseline, which we refer to as Grid-of-Pizels (GOP).
UncertalNR Alternatively, in this work, we parameterize f as an INR fo with parameters RP, map- ping from pixel coordinates (z. to pixel values fo(z, y). A reasonable prior for an independent and
TThe end-to-end INR approach introduced Tancik measurement. model.
2020 can thus be viewed maximum likelihood using Eq. 5
5

---

Published in Transactions on Machine Learning Research (04/2023)
INR with Uncertainty
Predicted Distribution
Uncerta
fa(x)192) y0( fo(xxn Ma)
X
f(x,))
Radon
Transform
L
Measurement Data
Predicted Sinogram
Mean Image
Figure UncertaINR Architecture: An end-to-end INR with UQ (BBB, MCD, HMC, and/or DEs) is sampled across all image pixels, generating distribution of predicted images, {,a, J J The predicted sinogram, generated by the Radon transform of predicted mean image f. is compared to the true measurement data in the INR. loss, L. Once training completes, f is reported as the reconstructed image and the predictive distribution across INR samples is used to quantify uncertainty.
identically-distribuue zero-mean Gaussian prior, (0) This gives rise to an implicit prior distribution over functions Jo- Such prior is standard in the Bayesian Deep Learning (BDL) literature (Blundell et al. 2015 Graves 2011], and is well- to yield Gaussian Process (GP) limit over fe for wide NNs (Neal 1996;
Matthews et al. 2018: Lee et al. 2018). However, properties of implicit parameter priors are less understood for finite-width NNs, even in standard BDL applications like image classification (due to the uninformative nature of NN parameter spaces), and much less so for CT reconstruction. Thus, we choose a composite prior for UncertaINR,
p(0) < px(())r(0) (0) = exp(-XI(fo))
(7)
combining (O),), which constrains the NN parameter values, with T(fa), which imposes a smooth regu- larization constraint on implicit images fo parameterized by We adopt the common medical imaging practice of using TV regularization, T Trv (Eq, [ To the best of our knowledge, this constitutes the first application of TV regularization to INRs, which can be seen in Appendix G.2 Table to noticcably improve UncertaINR reconstruction performance.
Inference Our overall framework for UncortaINR illustrated in Figure 2 Given parameter prior p(0),
model fo, and likelihood model p(S]Jo) from Eq. 5 we apply Bayes' rule to p(O x p(S),o)p(0), deriving the parameter space posterior distribution p(0SS). Given a set of sinogram measurements S, we ideally seek
to sample from the posterior
[ Σ
1
XTrv(6o)
P(0SS) exp
(S. A+5)) 272 11012
(8)
20²
which assigns high probability images that strike balance between: 1) small error between reconstruc- tion fo's 's Radon transform and observed measurements S,, and 2) low regularization cost under prior p(0). 7² is a variance hyperparameter for the zero-mean Gaussian parameter prior (0), in Eq.
Given N posterior INR parameter samples, (0,) from Eq. we can then use the induced posterior image samples [/ to infer both image reconstruction and UQ over the reconstruction. For example, the reconstructed image can be estimated, e.g. by the posterior mean over locations (æ, y) as
1 N N
f(x,y) =
Σ Sa (I,9), n=l
(9)

---

Published in Transactions on Machine Learning Research (04/2023)
and the posterior predictive uncertainty can be characterized, e.g. through the predictive variance
N
0²(z. = N ((,,))
(10)
This can be used to visualize, as in Figures D and Bb, regions of varying model uncertainty and calculate uncertainty metrics such as coverage and calibration (Section 4.1
Approximate Inference Unfortunately, exact inference in BDL is usually intractable due in part to the high-dimensional and complicated nature of NN posterior landscapes meaning one must resort to approximate inference methods. Several BDL algorithms have been proposed in the literature, based on approximations with varying levels of sophistication and implementation complexity. In our experiments, we consider four approaches: Bayes-By- Byy-accpoppppp (BBB), Monte Carlo Dropout (MCD), Hamiltonian Monte Carlo (HMC), and deep ensembles (DEs). BBB is variational evidence lower bound minimization proce- dure based on stochastic gradient descent (Blundell et al. 2015). MCD uses samples tost-time from NNs trained with dropout (Srivastava et al. 2014) (motivated
variational approxination of a deep Gaussian process GGal & Ghahramani ). HMC (Neal et al. 2011) is gold-standard yet more computationally expensive Markov Chain Monte Carlo (MCMC) procedure leveraging Hamiltonian dynamics (via a time- reversible and volume- preserving integrator) to better explore the full distribution typical set, decreasing consecutive sample correlation and reducing the number of samples required for convergence the pos- Duane et al. 1987). Finally DEs quantify predictive uncertainty by ensembling the outputs of several NN "base learners" " trained for the same task with different random sceds (Laksh- minarayanan et al., 2017), and is generally regarded as state-of-the-art approach for UQ in NNs (Ovadia et al. 2019). Despite non-Bayesian motivation in [akshminarayanan et al. (2017), the relationship be- tween Bayesian inference and DEs is an active area of research in the BDL community. Wilson & Izmailov
terior (Betancourt 2017
2020)) argue that DEs provide a more compelling approximation to the true posterior than many standard
BDL approaches, whilst others have adapted DEs to provide a Bayesian interpretation (Ciosek et al. 2019
D' D'Angelo & Fortuin
2021 Pearce et al.. 2020 He al. 2020).
Implementation Details
We note that ensembling can be combined with any of the three prior BDL meth- ods to further improve performance and, in particular, we found that ensembling MCD base learners provides additional gains, in practice, with UncertaINR. Surprisingly, our experiments show that MCD, arguably the simplest BDL approach, produces high-quality reconstructions, outperforming DEs and comparable results to the complex HMC procedure. MCD trains model fo to minimize the unnormalized version of Eq. X with dropout (Srivastava et al.) 2014) before every weight layer. The samples (Jo,), Jn of Eqs. g 10 are obtained by simply performing dropout at inference time, i.e. disabling each internal network node according to a pre-set probability for different random sceds and applying a forward pass over image coordinates (I, y) to obtain the pixel values fo., (z. y). Hence, the MCD samples from the approximate posterior can be obtained very efficiently after training. For HMC, we used the Tu----urr-ammller (Hoffman al.j 2014) sampling scheme in NumPyro (Phan et al., 22019), both for Une,rtaLNR, and also to obtain UQ with GOP. Further details of the approximate inference algorithms consider are given in Appendix D
4
Experimental Results
Project code is available at: https://github. httpa///...ooooobbyymm/mtttttttt
4.1
Experimental set up
Two datasets were considered. The first consists of artificial 256 x 256 pixel Shepp-Logan phan- tom (Shepp & Logan 1974] brain images and was used for ablations. Given the data's simple nature, ablations were performed in the extremely low measurement settings of 5- and 20-view sinograms. We tuned hyperparameters on 5 validation images and evaluated performance on test images. The second dataset, used for UncertaINR baseline comparisons, contains 512 x 512 pixel abdominal CT scan images, provided by
Datasets

---

Published in Transactions on Machine Learning Research (04/2023)
the Mayo Clinic for the 2016 Low-Dose CT AAPM Grand Challenge (MCoollugh et al. 2017 validation images and 8 test images were used to generate noisy 60- and 120-view sinograms
Performance Metrics We assessed reconstructed image samples {fe, J via four metrics: peak-signal- lo-noise ratão (PSNR), signal-to-noise ratio (SNR), negative log-likelihood (NI..), and expected colibration error (ECE). PSNR and SNR are common measures of predictive accuracy, whose equations we present in C NLL is common probabilistic model quality metric, assessing both predictive accuracy and uncertainty calibration, Under the assumption of an independent Gaussian model, each pixel's averaged prediction f(r y) (Eq. () is sampled from Gaussian distribution with the ground truth pixel value f" v) mean and calculated predicted variance of pixel responses, 6² (²(x,, y) from Eq. 10 variance. The NLL is the negative log- -igelikel of the pixel values under this model,
Appendix
E ( (f(x,v) - f=,, + log(2no"(z, a.y.
Ideally, the model maximizes the likelihood, meaning NLL is minimized when f(z, y) = fx y) for all (z, y) and variance (x,, y) is small.
Finally, ECE assesses model prediction of its outcome probabilities, gauging reliability of the model's pre- diction confidence. Specifically, it describes the discrepancy between the target coverage (TC) and achieved coverage (AC). Given N image samples Jim. each pixel (x, has an empirical distribution of N pre- dicted values, (z( v) {fa. (a, y) Ideally, the median of this distribution is the ground truth pixel value, f (x, y). For a given TC we define,
(xx.r=((1,1)) = i Q50- (Pw(x.w)
(Prtet) S l (2(xx)) S Q50+ (Bx(a.w))
(11)
where (F) denotes the kth quantile of distribution F. AC is thus defined as the percentage of pixel distributions containing (xx,8) y) in that quantile,
ACI AC.",,N.DD) 1x V1
C (oo.sglai (2,
(12)
2.81
If a model is perfectly calibrated, p% of the reconstructed pixel distributions will contain ("x y) in their p% quantile, meaning AC=TC for all quantiles. Given a finite set, P, of percentages evenly spaced in [o, 1]. ECE is defined as the average difference between AC and TC,
AC.F",N.PP) ECE(F" = P
Σ PEP
(13)
A reliability curve, as visualized in Figure 30, plots AC as a function TC. Better model calibration produces curves similar to the identity function, AC(f" FN, p) p. Furthermore, akin to inverse transform sampling, the marginal distribution of the inverse quantiles of calibrated pixel predictive distributions Fx. v), at ground truth pixel values (z( v), should be uniformly distributed in [0, 1]. We visualize such coverage hislogrums for UncertaINR in Figure For a further discussion of calibration, coverage, and implementation details, see Appendix C
4.2 Experiments on Artificial (Shepp-Logan) Data
Ablations
Experiments were performed on the artificial Shepp-Logan dataset, described in Section 4.1 to
ablate different UncortaINR hyperparameters across BDL approaches. We ablated the activation function (Tanh, SoftPlus, Sine, SiLU, and ReLU), depth, width, and random Fourier feature (RFF) embedding frequency. For MCD we also assessed sensitivity to the dropout probability p, while for BBB we ablated prior standard deviation and KL factor. For the sake of brevity, we only report main findings here (detailed analysis
3 Gannssian naise was added to achieve 40dB SNR relative original, noiseless sinogram.
8

---

Published in Transactions on Machine Learning Research (04/2023)
Table 1: Ablation Study: UncertaINR accuracy, relative classical approaches, and calibration results on
the Shepp-Logan phantom dataset. Results averaged across 5 validation and test imagos, with the best result for each metric (PSNR, NLL, and ECE) bolded.
RECONSTRUCTION
5-View VALIATTOON SET
5-View TEST SET
20-View VALIDATION SET
20-View TEST SET
TYPE
PSNR (1)
NLL (1)
ECE
PSNR (1)
NLL (1)
ECE
PSNR (f)
NLL
BCE
PSNR
NLL
ECE
FBP
7.68
5.15
17:35
15.71
CGLS
16.38
14.02
21.85
20.82
EM
19.88
30.22
29.11
SART
21.12
19.75
31.97
30.45
SIRT
21.12
21.12
31.98
30.44
UINR
0.562
0.119
MCD UINR
26.15
-1.473
0.111
24.45
-1.572
0.083
33.74
0.701
0.135
33.08
1.093
0.113
DR-2 MCD UINR
26.31
-1.730
0.081
24.40
-1.774
0.0G9
33.96
0.005
0.136
33.44
-0.372
0.102
MCD
-1.737
0.085
-1.751
0.067
34.31
-0.363
0.134
-0.625
0:101
DE-10 MCD UINR
26.36
-2.226
0.075
24.67
-1.960
0.068
34.38
-0.529
0.131
33. 36
-0.774
0.096
is provided in Appendix F.5 Activation function and RFF frequency were found to be the most critical hyperparameters. Sine activation produced the best-performing models, but resulting networks were sensitive to hyperparameter choice. SILU, ReLU, and Tanh achieved slightly lower, but more consistent reconstruction accuracies. In line with recent work demonstrating RFF importance for learning high-frequency image components (Tancik et al., 2020), we found that RFF frequency significantly affected model performance. Specifically, RFF frequency must. be consistent with the number of view angles too low (high) an RFF frequency leads to blurry images nhigh-frequence image artifacts).
Model comparison We also used the Shepp-Logar dataset to compare lowcomppexxtty reconstruction and UQ methods - including UncertaINR with BBB, MCD, and DEs of (2, 5, and 10) MCD base learners, as well as classical reconstruction baselines (FBP, CGLS, EM, SART, and SIRT). For the latter, without UQ, only reconstruction accuracy is reported. Several conclusions can be drawn from the experimental results summarized in Table with further analysis on variability and significance presented in Appendix F.7 First, MCD UINRs significantly outperform BBB UINRs, c.g. with PSNR gains up to 5dB (20-view test set) and ECE reductions by 2/a (5-view lest sel). Due to this poor performance (more detail in Appendix F. BBB UINRs were not considered in later experiments. Second, all MCD-based methods significantly outperformed classical mothods in terms of roconstruction accuracy, with gains up to 44dB PSNR in both test sets. Third, among UncertaINR methods, DE-5 and DE-10 10 MCD UINRs achieved the best performance, but the simple MCD UINR was surprisingly close (within 1dB PSNR of the best ensemble in all cases). Similar conclusions can be drawn with respect to UQ Furthermore, despite small validation set size (5 images), UINRs generalized well to the test set. For 20-views, validation and test accuracies were comparable, with improved test set calibration. For 5-views, despite slight PSNR and NLL degradation, ECE improved in the test set. These results suggest that small validation sets are sufficient for tuning UncortaINRs.
4.3 Experiments on Real-World (AAPM Grand Challenge) Data
We next compared UncertaINR to state-of-the-ars reconstruction approaches, on the real-world AAPM dataset, described in Section 4.1 Results are reported in Table 2 with further analysis on variability and significance in Appendix G.3 UncertaINRs were trained with MCD, DEs of INRs, DEs of MCD UINRs, and
HMC- all implemented with TV regularization. More information ion about dataset and model hyperparameters is given in Appendix G To understand the effect of UQ on reconstruction accuracy, we implemented our end-to-end INR. without UQ ("INR" in Table 2), similar to Tancik et al. (2020)'s 's proposal. For 60-vicws, MCD UINRs, DE UINRs and DE MCD UINRs outperformed INRs, whereas HMC UINRs were competilive, but underperformed. Overall, adding UQ did not harm INR reconstruction accuracy.
The lowest block of Table 2 presents results of CNNs trained on large datasets: FBP-UNet, GM-RED and these methods with ColL. We emphasize that these methods are trained on large training datasets, while INRs require only a handful of images for hyperparameter tuning. Thus, the results of these methods
Note that, while the COIL work leveraged GM-RED trained with deep image denoisers, RED can be used with denolsers that require less training data.
9

---

Published in Transactions on Machine Learning Research (04/2023)
provide upper bound on the reconstruction accuracy expected of UncertaINRs, but the two are not
directly comparable.
Nevertheless, all (U)INRs achieve competitive performance, reaching accuracies within 1dB (60-views) or 1.5dB (120-views) of the best (large training dataset) CNN. In fact, in the low-measurement regime (60- views), all (U)INRs achieved competitive performance with the highest reconstruction accuracy UINR (DF-5 MCD UINR) which outperformed all methods, except FBP-UNET with ColL.
The top 3 blocks of Table 2 present results for meth- ods that can be fairly compared to the (U)INRs: 1) classical FBP, CGLS, EM, SART, and SIRT meth- ods, 2) Sun et al. (2021)'s results for FISTA-TV, FBP with ColL, and FISTA-TV with ColL, and 3) TV regularized GOP with and without HMC. Similarly to the previously presented results for synthetic Shepp-Logan data, most classical meth- ods underperform the INRs by more than 5dB AAPM. Only FISTA-TV is competitive, albeit still more than 1dB away from the INRs. While adding ColL to FBP and FISTA-TV improves performance, these methods still perform worse than all 60-view (U)INRs. Furthermore, unlike ColL-based meth- ods, UnC is end-to-end and does not require pre-processing. Finally, we found the discretized GOP instatiation significantly underperformed all (U)INR methods- - highlighting the power of INRs relative to classical grid/voxo approaches.
Table 2: Performance Assessment: Accuracy and cal- ibration results of all approaches are presented for the AAPM dataset, with noise added to achieve a 40dB SNR. sinogram. The table is divided into 4 method types: classical, GOP, INRs, and DL. (*) denotes re- sults taken directly from (Sun et al. 2021]. Results are averaged over all 8 test set images and the best re- sult for each metric (SNR, NLL, and ECE) is bolded (excluding italicized DL methods, which rely on sub- stantially more training data, constituting an unfair UINR comparison).
RECONSTRUCTION
60- VIEWS
120-VIEW VIeWs
METIaN
SNR
NLE
RCE
SNR
NLL
ECE
FBP
10.58
14.11
EM
14.47
15.55
CGLS
20.08
21.94
SHRT
20.89
21.36
SART
21.54
21.77
FISTA-TV
26.08
27.59
ColL*
23.48
24.52
In terms of uncertainty calibration, Table 2 results are surprising. Contrary to common BDL intuition that DEs achieve the best model calibration (Oaaiia et al. 2,19), we found MCD to be more effective for INR UQ calibration. Specifically, MCD UINRs and DEs of MCD UINRs achieved significantly better model calibration than DEs of INRs without uncer- tainty. For example, introducing MCD to a DE-10 CINR reduced ECE from 0.144 to 0.045 (60-views) and from 0.162 to 0.053 (120-views). Although in- creasing ensemble sizes generally improved model calibration, the performance boost was not as sig- nificant as that of using MCD. We do not currently
TV ColL.
26.95
28.95
GOP-TV
25.97
27.40
HMC GOP-TV
25.10
-2.604
0.102
26.82
-3. 33.367
0.102
INR
27.25
28.81
DE-2 UINR
27.29
24.55
0.224
28.83
18
0.222
DE-5 UINR
27.30
6.427
0.176
28.83
8.804
0.183
DE-10 UINR
27.28
6.882
0.1444
28.82
6.346
0.162
MCD UINR
27.38
-3.447
0.078
28.65
-3.759
0.071
DE-2 MCD UINR
27.44
-3.573
0.066
28.68
-3.819
0.056
DE-5 MCD UINR
-3.660
28.70
-3.876
0.043
DE-10 MCD UINR
27.16
-3.689
0.015
28.74
-1.090
0.053
HMC UINR
27.10
-3.963
0.074
28.50
-4.021
0.085
FBP-UNKY*
27.08
29.18
GM-RED
27.12
29.30
FBP-UNET ColL.
27.93
29.71
GM-RED COIL"
27.42
29.79
have a full understanding of why DEs perform poorly relative to MCD for UINRs, in contrast to standard BDL applications. However, in Appendix F.6 we hypothesize that this may be due to the model capacity of an individual ensemble member, which is dictated through the RFF encoding frequency 2g.
Furthermore, although HMC is often considered a "golden standard" of approximate Bayesian inference Iz- mailov et al.) 2021), MCD UINRs achieved better performance than HMC GOP and HMC UINRs. While HMC UINRS achieved NLL competitive with the best DEs of MCD UINRs, they only achieved ECE compet- itive with the MCD UINR and noticeably lower SNR than all (U)INR approaches. Similar to observations in the Bayesian deep learning literature (Wenzel et al. 2020) and discussed in Appendix G we found a related cold posterior effect in which modifying posterior temperature enables either improved SNR across HMC samples or improved UQ calibration, but not both. In all, given the large computational overhead in tuning and training HMC relative to MCD, MCD DEs appear to offer the best compromise between computational speed, reconstruction accuracy, and well-calibrated uncertainty.
10

---

Published in Transactions on Machine Learning Research (04/2023)
HMC GOP
DE-10 UINR
MCD UINR
DE-5 MCD UINR
DE MCD UINR
HMC UINR
Reconstrucedd Mean
HMC 60P
DE-2
Abs. Error
Absolute Error
DE-10
UINR
0001
Variance
MCD
0003
UINR
DE-2
MCD
Coverage
DE-5
UINR
DE-10
MCD
UINR
Reliablity
6.2
HMC
UINR
1.000
1.0.0
01.
1.000
3.000
0.5
3.000
0.5
1.0
0.00
0.01
0.02
0.03
Target
Target Coveagg
Target Coverage
Target Coverage
Target Coverage
Target Coverage
(a) Calibration
(b) Reconstruction and Uncertainty Quantification on AAPM
Figure 3: For AAPM test image and the UQ approaches of this work: (a) Violin plots of pixel-wise absolute error and predicted standard deviation distributions indicate model calibration. (b) Visualizations of the reconstructed mean, absolute error, predicted variance, coverage, and reliability.
The benefits of MCD for INR calibration are further illustrated by the violin plots of Figure Ba, comparing the pixel-wisc absolute error versus predicted standard deviation distributions across GOP and UINR models. For non-MCD DEs, the standard deviation distribution skews towards smaller values than the absolute error distribution, indicating that the model overconfident. The opposite is true for HMC GOP, which is underconfident.. Meanwhile, HMC and MCD UINRs predicted standard deviation distributions most closely resembling those of the absolute error, indicating decent calibration.
Finally, Figure 3p visualizes the model output, calibration diagnostics, and uncertainty on single test image for GOP and the different UINRs proposed in this work. Similar figures are presented for the remaining test images in Appendix G As reflected in the absolute error images, the GOP reconstructed mean is blurry relative to those of the UINRs. Meanwhile, the reliability curves and coverage plots of both HMC GOP and the DE-10 UINR are quite poor relative the nearly uniform coverage plots and ideal reliability curves of the (DE) MCD UINRs and HMC UINR. However, these absolute error, coverage, and reliability metrics require the ground truth image to compute and thus would not be available to doctor.
In real-world scenarios, the only visualizations available are the reconstructed mean and variance images. Given that a well-calibrated model reports larger variance in regions of larger absolute error, variance can used as a proxy for reconstruction error. For example, the Figure 3p absolute error images show that the models underperform at prodicting boundaries between different tissues, which reflected in corresponding higher uncertainties in the variance images. When presented doctor, our UncertaINR variance images could inform more cautious diagnoses based perceived issues in those regions.
11

---

Published in Transactions on Machine Learning Research (04/2023)
5 Limitations, Broader Impact, and Future Work
Despite its decent performance, UncertaINR should not be considered as replacement for professional medical diagnosis but simply two datasets, with AAPM data representing a retrospetivell-simnlice low-dose acquisition. While these experiments offer a promising proof-of-concept of uncertainty quantification for INR-based low-dose CT reconstruction, additional evaluation should be performed on larger, real acquired low-dose CT data before this strategy deployed medical settings.
tool to aid it. In this work, UncertaINR was thoroughly studied on
Possible future work includes evaluating UncertaINR in other modalities, such MRI or 3D/4D imaging settings. In such extensions it would be beneficial to further improve training and memory efficiency of UncertaINR, and to do so UncertaINR could be combined with recent INR-related advances, such meta- learning (Tancik ct al.) 2021p and compression (Dupont et al. 2021). Another possibility is to leverage the predictive uncertainty achieved by UINR in an active learning (Cohn et al.) 1996) setting, enabling efficient measurement procedures and reducing harmful patient radiation exposure. Finally, our findings raise some fundamental questions about UQ in the INR setting, such as the poor performance of deep ensembles and the effectiveness of MCD, which counter common beliefs in, and should be of interest to, the Bayesian deep learning community.
6 Conclusion
In this work, we proposed UncertaINR: a Bayesian reformulation of INR-based image reconstruction. In the high-stakes and well-motivated application of CT image reconstruction, UncertaINR attained calibrated uncertainty estimates without sacrificing reconstruction quality relative to other classical, INR-based, and CNN-based roconstruction techniques on rcalistic, noisy, and underdetermined data. In the context of INR. UQ, contrary common intuition, we found that simple and efficient MC Dropout rivaled (even outperformed) the popular deep ensembles and the sophisticated, yet computatiionally-xxpnnn Monte Carlo methods. UncertalNR's strategic use of INRs outperformed classical reconstruction approaches, while alleviating key challenges faced by state-of-the-ar DL methods namely generalizability and small- scale medical datasets. In addition to informing doctor diagnoses, UncertaINR's well-calibrated uncertainty estimates could pave the way for reduced healtheare costs, via mothods like automated triage, and reducod patient radiation exposure, via methods like active learning.
Hamiltonian
7 Acknowledgements
Francisca Vasconeelos primarily carried out this work the University of Oxford, while supported by the Rhodes Trust via a Rhodes Scholarship. She is now at UC Berkeley, supported by an NSF Graduate Research Fellowship under grant number DGE 2146752. Bobby He is supported by the EPSRC and MRC through the OxWaSP CDT programme (FP/L.01610/11) Nalini Singh is supported by a Google PhD Fellowship.
References
Michael Abramoff, Philip Lavin, Michele Birch, Nilay Shah, and James C Folk. Pivotal Trial of an Autonomous Al-based Diagnostic System for Detection of Diabetic Retinopathy in Primary Care Offices. NPJ digital medicine, 111):1-8, 2018.
Ben Adlam, Jasper Snock, and Samuol Smith. Cold Posteriors and Alcatoric Uncertainty. arXiv preprint arXiv:2008.0000 00029, 2020.
Jonas Adler and Ozan Öktem. Learned Primal-Dual Reconstruction. IEEE transactions on medical imaging, 37(6):1322-1332 2018.
Laurence Aitchison. A Statistical Theory of Cold Posteriors in Deep Neural Networks. In International Conference Learning Represendations, 2020.
12

---

Published in Transactions on Machine Learning Research (04/2023)
Anders H. Andersen and Avinash C. Kak. Simultaneous Algebraic Reconstruction Technique (SART): A
Superior Implementation of the ART Algorithm. Ultrasonic Imaging, 6(1):81-99) -94, 1984.
Riccardo Barbano, Javier Antoran, José Miguel Hernández-Lobato and Bangti Jin. Probabilistic Deep Image Prior over Image Space. In Fourth Symposium on Advances in Appronimate Bagesion Inference, 2021.
Amir Beck and Marc Teboulle. Fast Iterative Shrinkage-Thresholciin Algorithm for Linear Inverse Prob- lems. SIAM journal on imaging sciences, 2(1):183-22) 202, 2009.
M.J. Berger, J.S. Coursey, M.A. Zucker, and J. Chang. ESTAR, PSTAR, and ASTAR: Computer Programs for Calculating Slopping-Power and Range Tables for Electrons, Prolons, and Helium Tons. National
Institute of Standards and Technology, Gaithersburg, MD, 2004. (version 1.2.3) Availabile: http://
physics wws.. gov/Str [2021, 08, 22). Originally published as: Berger, M.J., NISTIR 4999, National Institute of Standards and Technology; Gaithersburg, MD (1993).
Michael Betancourt. A Conceptual Introduction to Hamiltonian Monte Carlo.
arXiv preprint
arXiv:1701.02444 2017.
Lukas Biewald. Experiment Tracking with Weights and Biases, 2020. URL attps://wwy wandb. com/ Software available from wandb.com.
Charles Blundell, Julien Cornebise, Koray Kayukeuoglu, and Daan Wierstra. Weight Uncertainty in Neural Network. In International Conference Machine Learning, pp. 1613-1622. PMLR, 2015.
Ronald N Bracewell. Strip Integration in Radio Astronomy. Australian Journal of Physics, 9(2):198-2 1977117 1956.
James Bradbury, Roy Frostig, Peter Hawkins, Matthew James Johnson, Chris Leary, Dougal Maclaurin, George Necula, Adam Paszke, Jake VanderPlas, Skye Wanderman-Milne, and Qiao Zhang. JAX: compos- able transformations of Python+NumPPy programs, 2018. URL http::/gitthe com/ coom/goolee/jp
David J. Brenner and Eric J. Hall. Computed Tomography - An Increasing Source of Radiation Exposure. New England Journal of Medicine, 357(22): 2007.
David R. Bull. Chapter Digital Picture Formats and Representations. In David R. Bull (ed.), Commun
nicating Pictures, pp. 99 132. Academic Press, Oxford, 2014. ISBN 778-0-12-405906-11
Gary S. Bust and Cathryn N. Mitchell. History, Current State, and Future Directions of lonospheric Imaging. Reviews of Geophysics, 46(1), 2008.
Hu Chen, Yi Zhang, Mannudeep K Kalra, Feng Lin, Yang Chen, Peixi Liao, Jiliu Zhou, and Ge Wang, Low-dosc CT with a Residual medical imaging, 36(12):2524-2588) 2017a.
Convolutional Neural Network. IEEE transactions on
Hu Chen, Yi Zhang, Weihua Zhang, Peixi Liao, Ke Li, Jiliu Zhou, and Ge Wang. Low-dose CT via Convo- lutional Neural Network. Biomedical optics express, 8(2):679-66) 694, 2017b.
Zhiqin Chen and Hao Zhang. Learning Implicit Fields for Generative Shape Modeling. In Proceedings of the IEEF/CVF Conference on Computer Vision and Pattern Recognation, pp. 5939 5948, 2019.
Siddhartha Chib and Edward Greenberg. Understanding the Metropolis Hastliss-astigsss Algorithm. The american
statistician, 49)4:3327-38 -335, 1995.
Kamil Ciosek, Vincent Fortuin, Ryota Tomioka, Katja Hofmann, and Richard Turner. Conservative certainty estimation by fitting prior networks. In International Conference on Learning Representations, 2019.
David A. Cohn, Zoubin Ghahramani, and Michael I. Jordan. Active Learning with Statistical Models. Journal of Artificial Intelligence Research, 21199-455 1996.
13

---

Published in Transactions on Machine Learning Research (04/2023)
Beau Coker, Wessel P Bruinsma, David R Burt, Weiwei Pan, and Finale Doshi- Wide Mcan-Fied Bayesian Neural Networks Ignore the Data. In International Conference Artificial Intelligence and Statistics, pp. 5276 5333. PMLR, 2022.
George Cybenko. Approximation by Superpositions of a Sigmoidal Function. Mathematics of Control, Signals and Systems, 2(4))303 314, 1989.
Francesco D' D'Angelo and Vincent. Fortuin. Repulsive Deep Ensembles are Bayesian. In A. Beygelzimer, Y. Dauphin, P. Liang, and J. Wortman Vaughan (eds.), Advances in Neural Information Processing Sys- tems, 2021.
Jeffrey De Fauw, Joseph R Ledsam, Bernardino Romera-Paredes, Stanislav Nikolov, Nenad Tomasev, Sam Blackwell, Harry Askham, Xavier Glorot, Brendan O'Donoghue, Daniel Visentin, et al. Clinically Ap- plicable Deep Learning for Diagnosis and Referral in Retinal Disease. Nature medicine, 24(9):1342-1350, 2018.
Amy Berrington de Gonzalez and Sarah Darby. Risk of Cancer from Diagnostic X-Rays: Estimates for the UK and 14 Other Countries. The Lancel, (6306)6))3345 351, 2004.
Amy Berrington De González, Mahadevappa Mahesh, Kwang-Pye Kim, Mythreyi Bhargavan, Rebecca Lewis, Fred Mettler, and Charles Land. Projected Cancer Risks from Computed Tomographic Scans Performed in the United States in 2007. Archives of Internal Medicine, 169(22):2071 2077, 2009.
Stanley R. Deans. The Radon Transform and Some of Its Applications. Courier Corporation, 2007.
Boyang Deng, John P Lewis, Timothy Jeruzalski, Gerard Pons-Moll, Geoffrey Hinton, Mohammad Norouzi, and Andrea Tagliasacchi. NASA: Neural Articulated Shape Approxination. In Compuler Eiton--CCV 2020: 16th European Conference, Glasgow, UK, August 23 28, 2020, Proceedings, Part VII 16, pp. 612 628. Springer, 2020.
Jia Deng, Wei Dong, Richard Socher, Li-Jia Li, Kai Li, and Li Fei-Fei. Imagenet: A Large- Scale Hierarchical Image Database. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 248-24282 248-255, 2009.
Armen Der Kiureghian and Ove Ditlevsen. Alcatory or Epistemic? Does It Matter? Structural Safety, 31 (2):105-112, 2009.
Bao-Yu Dong. Image Reconstruction using EM Method in X-Ray CT. In 2007 International Conference on Wavelet Analysis and Pullern Recognition, volume 1, pp. 130 134. TEEE, 2007.
Simon Duane, A.D. Kennedy, Brian J. Pendletoon, and Duncan Roweth. Hybrid Monte Carlo. Physics Lellers B, 195(2):216-22)) 222, 1987. ISSN 0370-2693.
Charles Dugas, Yoshua Bengio, François Bélisle, Claude Nadcau, and René Garcia. Incorporating Sccond- Order Functional Knowledge for Better Option Pricing, Advances in Neural Information Processing Sys- lems, pp. 472-478, 2001.
Fmillen Dupont, Adam Golinski, Milad Alizadeh, Yee Whye Teh, and Arnaud Doucet. COIN: COmpression with Implicit Neural representations. In Neural Compression: From Information Theory Applications WWorkshop @ ICLR 2021, 2021.
Emilien Dupont, Hyunjik Kim, S. M. Ali Eslami, Danilo Jimenez Rezende, and Dan Rosenbaum. From data to functa: Your data point is function and you can treat it like one. In Proceedings of the 39th International Conference on Machine Learning, volume 162 of Proceedings of Machine Learning Research, pp. 5694 5725. PMLR, 17 23 Jul 2022a.
Emilien Dupont, Yee Whye Teh, and Arnaud Doucet. Generative Models as Distributions of Functions. The 25th International Conference Artificial Intelligence and Statistics, 2022b.
14

---

Published in Transactions on Machine Learning Research (04/2023)
Piero Esposito. BLiTZ Bayesian Layers in Torch Zoo (a Bayesian Deep Learing library for Torch). https: 2020.
Vincent Fortuin, Adrià Garriga-Alonso, Sebastian W. Ober, Florian Wenzel, Gunnar Ratsch, Richard E Turner, Mark van der Wilk, and Laurence Aitchison. Bayesian Neural Network Priors Revisited. Tn International Conference on Learning Representations, 2022.
Sara Fridovich-Keil, Alex Yu, Matthew Tancik, Qinhong Chen, Benjamin Recht, and Angjoo Kanazawa. Plenoxels: Radiance Fields Without Neural Networks. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognation, pp. 5501-5510, 2022.
Yarin Gal and Zoubin Chahramani. Dropout Bayesian Approximation: Representing Model Uncertainty in Deep Learning. In International Conference on Machine Learning, pp. 1050-1059. PMLR, 2016.
Kylc Genova, Forrester Cole, Daniel Vlasic, Aaron Sarna, William T. Freeman, and Thomas Funkhouser. Learning Shape Templates with Structured Implicit Functions. In Proceedings of the IEEE International Conference on Computer Vision, pp. 1154-71644 2019.
Kyle Genova, Forrester Cole, Avneesh Sud, Aaron Sarna, and Thomas Funkhouser. Local Deep Implicit Functions for 3D Shape. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4857 4866, 2020.
Peter Gilbert. Tterative Methods for the Three-Dimensions Reconstruction of an Object from Projections. Journal of theoretical biology, 36(1):105-117, 1972.
Walter R. Gilks, Sylvia Richardson, and David Spiegelhalter. Markov Choin Monte Carlo in Practice. CRC Press, 1995.
Xavier Glorot, Antoine Bordes, and Yoshua Bengio. Deep Sparse Rectifier Neural Networks. In Proceedings of the Fourteenth International Conference on Artificial Intelligence and Statistics, pp. 315 323. JMLR Workshop and Conference Proceedings, 2011.
Ian Goodfellow, Yoshua Bengio, and Aaron Courville. Deep Learning. MIT Press, 2016.
Richard Gordon, Robert Bender, and Gabor T. Herman. Algebraic Reconstruction Techniques (ART) for Throo-Dimensional Electron Microscopy and X-Ray Photography. Journal of Theoretical Biology, 29(3):
471-481, 1970.
Alex Graves. Practical Variational Inference for Neural Networks. Advances in Neural Information Processing Systems, 24, 2011.
Chuan Guo, Gcoff Pleiss, Yu Sun, and Kilian Q. Weinberger. On Calibration of Modern Neural Networks. In Doina Precup and Yee Whye Teh (eds.), Proceedings of the 34th International Conference Machine Learning, volume 70 of Proceedings of Machine Learning Research, pp. 1321-1330. PMLR, 2017.
Boris Hanin. Universal Function Approximation by Deep Neural Nets with Bounded Width and ReLU Activations. Mathematics, 7(10), 2019. ISSN 2227-7390.
Kazuyuki Hara, Daisuke Saito, and Hayaru Shouno. Analysis of Function of Rectified Linear Unit used in Deep Learning. In 2015 International Joint Conference on Neural Networks (IJCNN), pp. 1-8, 2015.
Bobby He, Balaji Takshminarayanan, and Yee whye Teh. Bayesian Deep Ensembles via the Neural Tangent Kernel. In H. Larochelle, M. Ranzato, R. Hadsell, M. F. Balcan, and H. Lin (eds.), Advances in Neural Information Processing Systems, volume 33, pp. 1010-1022. Curran Associates, Inc., 2020.
Sigurdur Helgason. Groups & Geometric Analysis: Radon Transforms, Invariant Differential Operators and Spherical Panctions: Volume 1. Academic Press, 1984.
Philipp Henzler, Niloy Mitra, and Tobias Ritschel. Learning a Neural 3d Texture Space from 2D Exemplars. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 8356-8368 8364, 2020.
15

---

Published in Transactions on Machine Learning Research (04/2023)
Matthew D Hoffman, Andrew Gelman, et al. The No-U-Turn sampler: adaptively setting path lengths in Hamiltonian Monte Carlo. J. Mach. Learn. Res., 15(1):1593-162) 2014.
Kurt Hornik. Approximation Capabilities of Multilayer Feedforward Networks. Neural Networks, 4(2): 251-257, 1991.
Kurt Hornik, Maxwell Stinchcombe, and Halbert White. Multilayer Feedforward Networks are Universal Approximators. Neural Networks, 2(5):359 366, 1989.
Ahmed Hosny, Chintan Parmar, John Quackenbush, Lawrence H Schwartz, and Hugo JWL Aerts. Artificial Intelligence in Radiology. Nature Reviews Cancer, 18(8):500 -115000-11,,,0 2018.
Jiang Hsich. Computed Tomography: Principles, Design, Artifacts, and Recent Advances, volume 114. SPIE Press, 2003.
J.H. Hubbell and S.M. Seltzer. Tables of X-Ray Mass Attenuation Coefficients and Mass Energy-Absorption Coefficients. National Institute of Standards and Technology, Gaithersburg, MD, 2004. (version 1.4) Avail- abile: http://physics. mist. www.sniiiittttaaaiieeoeeeeeeeeee gov/xaamdi [2021, 08, 22]. Originally published NISTIR 5632, National Institute of Standards and Technology, Gaithersburg, MD (1995).
Pavel Izmailov, Dmitri Podoprikhin, Timur Garipov, Dmitry Vetrov, and Andrew Gordon Wilson. Averaging Weights Leads to Wider Optima and Better Generalization. In 34th Conference on Uncertainty in Artificial Intelligence 2018, UAI 2018, pp. 876 885, 2018.
Pavel Izmailov, Sharad Vikram, Matthew D Hoffman, and Andrew Gordon Gordon Wilson What are Bayesian neural network posteriors really like? In International Conference on Machine Learning, pp. 4629-48 4640. PMLR, 2021.
Arthur Jacot, Franck Gabriel, and Clément Hongler. Neural Tangent Kernel: Convergence and General- ization in Neural Networks. In Proceedings of the 53rd Annual ACM SIGACT Symposium Theory of Compating, pp. 6 6, 2021.
Chiyu Jiang, Avneesh Sud, Ameesh Makadia, Jingwei Huang, Matthias NieBner, Thomas Funkhouser, et al. Local Implicit Grid Representations for 3D Scenes. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recogrition, pp. 6001 6010, 2020.
Kyong Hwan Jin, Michael McCann, Emmanuel Froustey, and Michael Unser. Deep Convolutional Neural Network for Inverse Problems in Imaging. IEFE Transactions Image Processing, 26(9):4509 (599:4500-4422 2017.
Alex Kendall and Yarin Gal. What Uncertainties Do We Need in Bayesian Deep Learning for Computer Vision? Advances in Newral Information Processing Systems, 30:5574 5584, 2017.
Muhammad Osama Khan and Yi Fang. Implicit Neural Representations for Medical Imaging Segmentation. Tn International Conference on Medical Image Computing and Computer-Assio Intervention, pp. 433- 443. Springer, 2022.
Patrick Kidger and Terry Lyons. Universal Approximation with Deep Narrow Networks. In Conference Learning Theory, pp. 2306-2327. PMLR, 2020.
Diederik P Kingma and Max Welling. Auto-Encoding Variational Bayes slat, 1050:1, 2014.
Volodymyr Kuleshov, Nathan Fenner, and Stefano Ermon. Accurate Uncertainties for Deep Learning using Calibrated Regression. In International Conference on Machine Learning, pp. 2796-2804. PMLR, 2018.
Balaji Lakshminarayanan, Alexander Pritzel, and Charles Blundell. Simple and Scalable Predictive Un- certainty Estimation using Deep Ensembles. Advances in Neural Information Processing Systems, 30, 2017.
16

---

Published in Transactions on Machine Learning Research (04/2023)
Mex-Heinrichh Laves, Malte Tölle, Alexander Schlaefer, and Sandy Engelhardt. Posterior Temperature Op- timized Bayesian Models for Inverse Problems in Medical Imaging. Medicol image analysis, 78:102382, 2022.
Jaehoon Lee, Yasaman Bahri, Roman Novak, Samuel $ Schoenholz, Jeffrey Pennington, and Jascha Sohl- Dickstein. Deep Neural Networks as Gaussian Processos. In International Conference on Learning Repre- sentalions, 2018.
Michael M Lell and Marc KachelrieB. Recent and Upcoming Technological Developments in Computed Tomography: High Speed, Low Dose, Deep Learning, Multienergy. Investigatioe radiology, 55(1):8 19, 2020.
Zhengqi Li, Simon Niklaus, Noah Snavely, and Oliver Wang. Neural Scene Flow Fields for Space- Spaco-Time Time View Synthesis of Dynamic Scenes. In Proceedings of the IEEE/CVP Conference Computer Vision and Pattern Recognition, pp. 0998-6508, 2021.
Eugene C Lin. Radiation Risk from Medical Imaging. Mayo Clinie proceedings, 85 12:1142 6; quiz 1146, 2010.
David B. Lindell, Julien NP Martel, and Gordon Wetzstein. AutoInt: Automatic Integration for Fast Neural Volume Rendering. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 14556-14565, 2021.
Yan Liu and Yi Zhang. Low-dose CT restoration via stacked sparse denoising autoencoders. Newrocompating, 284:80 89, 2018
LuShin Lu, Yanhui YeonjongSu, and George Em Karniadakis. Dying ReLU and Initialization: Theory and Numerical Examples. Communications in Computotional Physics, 2855):1671-1706 2020.
Zhou Lu, Hongming Pu, Feicheng Wang, Zhiqiang Hu, and Liwei Wang. The Expressive Power of Neural Networks: A View from the Width. In I. Guyon, U. V. Luxburg, S. Bengio, H. Wallach, R. Fergus, S. Vishwanathan, and R. Garnett (eds.), Advances in Neural Information Processing Systems, volume 30. Curran Associates, Inc., 2017.
Ricardo
Noha Radwan, Mehdi S.M. Sajjadi, Jonathan T. Barron, Alexey Dosovitskiy, and Daniel Duckworth. NeRF in the Wild: Neural Radiance Fields for Unconstrained Photo Collections. Tn Proceedings of the IEEE/CVF Conference Computer Vision and Pattern Recognition, pp. 7210-7219, 2021.
Alexander G de G Matthews, Jiri Hron, Mark Rowland, Richard E Turner, and Zoubin Ghahramani. Gaus- sian Process Behaviour in Wide Deep Neural Networks. In International Conference on Learning Repre- sentations, 2018.
Cynthia H McCollough, Adam C Bartley, Rickey E Carter, Baiyu Chen, Tammy A Drees, Phillip Edwards, David R Holmes III, Alice E Huang, Farhana Khan, Shuai Leng, et al. Low-dose CT for the detection and classification of metastatic liver lesions: results of the 2016 low dose CT grand challenge. Medical physics, (5(10))c339-3322 2017.
Lars Mescheder, Michael Occhsle, Michael Nicmeyer, Scbastian Nowozin, and Andreas Geiger. Occupancy Networks: Learning 3D Reconstruction in Function Space. In Proceedings of the IEEE/CVP Conference on Computer Vision and Pattern Recognition, pp.
2019.
L.M. Mescheder. Stability and Expressiveness of Deep Generative Models. Eberhard Karls Universität Tübingen, 2020.
Ben Mildenhall, Pratul P. Srinivasan, Matthew Tancik, Jonathan T. Barron, Ravi Ramamoorthi, and Ren Ng. NeRF: Representing Scenes as Neural Radiance Fields for View Synthesis. In Europeon Conference on Computer Vision, pp. 405-499 0154421. Springer, 2020.
17

---

Published in Transactions on Machine Learning Research (04/2023)
Thomas Müller, Alex Evans, Christoph Schied, and Alexander Keller. Instant Neural Graphies Primitives with a Multiresolution Hash Encoding. ACM Trans. Graph,, 41 11141:102111102:15 July 2022.
Seth Nabarro, Stoil Ganev, Adrià Garriga-Alonso, Vincent Fortuin, Mark van der Wilk, and Laurence Aitchison. Data augmentation in Bayesian neural networks and the cold posterior effect. In Proceedings of the Thirty- Highth Conference on Uncertainty in Artificial Intelligence, volume 180 of Proceedings of Machine Learning Research, pp. 1434-1444. PMLR, 01-05 Aug 2022.
Radford Neal. Priors for Infinite Networks. In Bayesian Learning for Neural Networks, pp. 29-53. Springer, 1996.
Radford M. Neal et al. MCMC using Hamiltonian Dynamics. Handbook of Markov Chain Monte Carlo, 2 (11):2, 2011.
Behnam Neyshabur, Zhiyuan Li, Srinadh Bhojanapalli, Yann LeCun, and Nathan Srebro. Towards Un- derstanding the Role of Over-Paramettizatioon in Generalization of Neural Networks. In International Conference on Learning Representations (ICLR), 2019.
Michael Niemeyer, Lars Mescheder, Michael Oechsle, and Andreas Geiger. Differentiable Volumetric Ren- dering: Learning Implicit 3D Representations without 3D Supervision. In Proceedings of the IEFE/CVP Conference on Computer Vision and Pattern Recognition, pp. 3504 3515, 2020.
Jeremy Nixon, Michael W. Dusenberry, Linchuan Zhang, Ghassen Jerfel, and Dustin Tran. Measuring Calibration in Deep Learning. In CVPR Workshops, volume 2(7), 2019.
Lorenzo Noci, Kevin Roth, Gregor Bachmann, Sebastian Nowozin, and Thomas Hofmann. Disentangling the Roles of Curation, Data-Augmentation and the Prior in the Cold Posterior Effect. Advances in Neural Information Processing Systems, 34, 2021.
Michael Oechsle, Lars Mescheder, Michael Niemeyer, Thilo Strauss, and Andreas Geiger. Texture Fields: Learning Texture Representations Function Space. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 4531-4540, 2019.
Yaniv Ovadia, Emily Fertig, Jie Ren, Zachary Nado, D Sculley, Sebastian Nowozin, Joshua Dillon, Balaji Lakshminarayanan, and Jasper Snoek. Can you trust your model's uncertainty? Evaluating predictive uncertainty under dataset shift. Advances in Neural Information Processing Systems, 32:13991 14002, 2019.
Xiaochuan Pan, Emil Y. Sidky, and Michael Vannier. Why Do Commercial CT Scanners Still Employ Traditional, Filtered Back-Projection for Image Reconstruction? Inverse Problems, 25(12): 2009.
Jeong Joon Park, Peter Florence, Julian Straub, Richard Newcombe, and Steven Lovegrove. DeepSDF: Learning Continuous Signed Distance Functions for Shape Representation. In Proceedings of the IEEE/CVP Conference on Computer Vision and Pattern Recognition, pp. 165-174, 2019.
Adam Paszke, Sam Gross, Francisco Massa, Adam Lerer, James Bradbury, Gregory Chanan, Trevor Killeen, Zeming Lin, Natalia Gimelshein, Luca Antiga, Alban Desmaison, Andreas Kopf, Edward Yang, Zachary DeVito, Martin Raison, Alykhan Tejani, Sasank Chilamkurthy, Benoit Steiner, Lu Fang, Junjie Bai, and Soumith Chintala. PyTorch: An Imperative Style, High-Performance Deep Learning Library. In H. Wallach, H. Larochelle, A. Beygelzimer, F. d' AAlché-Buc, E. Fox, and R. Garnett (eds.), Advances in Newral Information Processing Systems 32, pp. 8024 8035, Curran Associates, Inc., 2019.
Tim Pearce, Felix Leibfried, and Alexandra Brintrup. Uncertainty in Neural Networks: Approximately Bayesian Ensembling. In Proceedings of the Twenty Third International Conference on Artificial Intelli- gence and Statistics, volume 108 of Proceedings of Machine Learning Research, pp. 234-244, 26-28 Aug 2020.
Daniël M Pelt, Doga Gürsoy, Willem Jan Palenstijn, Jan Sijbers, Francesco De Carlo, and Koes Joost Batenburg. Integration of TomoPy and the ASTRA Toolbox for Advanced Processing and Reconstruction of Tomographic Synchrotron Data. Journal of Synchrotron Radialion, 23(3): :842 849, 2016.
18

---

Published in Transactions on Machine Learning Research (04/2023)
Du Phan, Neeraj Pradhan, and Martin Jankowiak. Composable Effects for Flexible and Accelerated Prob- abilistic Programming in NumPyro. arXiv preprint urxim:1913 1 1154,, 2019.
Eugenio Picano. Sustainability of Medical Imaging. Bmj, 328((439):578 580, 2004.
S.E. Pryse, L. Kersley, D.L. Rice, C.D. Russell, and L.K. Walker. Tomographic Imaging of the ionospheric Mid-Latitude Trough. Anmales Geophysicne, 11(2-3):144-144 1993.
Maithra Raghu, Ben Poole, Jon Kleinberg, Surya. Ganguli, and Jascha Sohl-Dickstein. On the Expressive Power of Deep Neural Networks. In International Conference on Machine Learning, pp. 2847- -854. PMLR, 2017.
Ali Rahimi and Benjamin Recht. Random Features for Large-Scale Kernel Machines. In Proceedings of the 20th International Conference on Neural Information Processing Systems, pp. 1177-1184, 2007.
Prajit Ramachandran, Barret Zoph, and Quoc V. Le. Searching for Activation Functions. arXic preprint SSl::17100059444 2017.
Albert W Reed, Hyojin Kim, Rushil Anirudh, Aditya Mohan, Kyle Champley, Jingu Kang, and Suren Jayasuriya. Dynamic CT Reconstruction from Limited Views with Implicit Neural Representations and Parametric Motion Fields. In Proceedings of the IEEE/CVF International Conference on Computer Vision,
pp. 2258 2268, 2021.
Danilo Jimenez Rezende, Shakir Mohamed, and Daan Wierstra. Stochastic Backpropagation and Approxi- mate Inference Deep Generative Models. In Proceedings of the 31st International Conference on Machine Learning, Proceedings of Machine Learning Research, pp. 1278-1286 2014.
C.A. Roobottom, G. Mitchell, and G. Morgan-Hughes. RadiationnReduutioon Strategies Cardiac Computed Tomographic Angiograph. Clinical Radiology, (5(1)):859 867, 2010. ISSN 0009-9260.
Leonid I Rudin, Stanley Osher, and Emad Fatemi. Nonlinear Total Variation Based Noise Removal Algo- rithms. Physica D: nonlinear phenomena, 60(1-4):259-2) 268, 1992.
Shunsuke Saito, Zeng Huang, Ryota Natsume, Shigeo Morishima, Angjoo Kanazawa, and Hao Li. PIFu: Pixel-Aligned Implicit Function for High-Resolution Clothed Human Digitization. In Proceedings of the IEEE International Conference on Computer Vision, pp. 2304-2314, 2019.
Lawrence A Shepp and Benjamin F Logan. The Fourier Reconstruction of a Head Section. IEEE Transactions on Nuclear Science, 21(3):21-43, 1974.
Vincent Sitzmann, Michael Zollhoefer, and Gordon Wetzstein. Scene Representation Networks: Continuous 3D-Structure-Aware Neural Scene Representations. Advances in Neural Information Processing Systems, 32:1121-1132, 2019.
Vincent Sitzmann, Julien Martel, Alexander Bergman, David Lindell, and Gordon Wetzstein. Implicit Neural Representations with Periodic Activation Functions. Advances in Neural Information Processing Systems, 33, 2020.
Rebecca Snith-Bindman, Jafi Lipson, Ralph Marcus, Kwang-Pyo Kim, Mahadevappa Mahesh, Robert Gould, Amy Berrington De González, and Diana L. Miglioretti. Radiation Dose Associated with Common Computed Tomography Examinations and the Associated Lifetime Attributable Risk of Cancer. Archives of Internal Medicine, (69():220078 2086, 2009.
Nitish Srivastava, Geoffrey Hinton, Alex Krizhevsky, Ilya Sutskever, and Ruslan Salakhutdinov. Dropout: A Simple Way to Prevent Neural Networks from Overfitting. Journal of Machine Learning Research, 15
156):1929-1958, 2014.
Yu Sun, Jiaming Liu, Mingyang Xie, Brendt Wohlberg, and Ulugbek S. Kamilov. CoIL: Coordinate-Based Internal Learning for Tomographic Imaging. IEEE Transactions on Computational Imaging, 7:1400-1412, 2021.
19

---

Published in Transactions on Machine Learning Research (04/2023)
Matthew Tancik, Pratul Srinivasan, Ben Mildenhall, Sara Fridovich-Keil. Nithin Raghavan, Utkarsh Singhal, Ravi Ramamoorthi, Jonathan Barron, and Ren Ng. Fourier Features Tet Networks Learn High Frequency Functions in Low Dimensional Domains. In H. Larochelle, M. Ranzato, R. Hadsell, M. F. Balcan, and H. Lin (eds.), Advances in Newral Information Processing Systems, volume 33, pp. 7537-7547, 2020.
Matthew Tancik, Ben Mildenhall, Terrance Wang, Divi Schmidt, Pratul P. Srinivasan, Jonathan T. Bar- ron, and Ren Ng. Learned Initializations for Optimizing Coordinate-Based Neural Representations. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 2021.
Matthew Tancik, Vincent Casser, Xinchen Yan, Sabeek Pradhan, Ben Mildenhall, Pratul P Srinivasan, Jonathan T Barron, and Henrik Kretzschmar. Block-NeRF:: Scalable Large Scene Neural View Synthesis. In Proceedings of the IEEE/CVE Conference on Computer Vision and Pattern Recognition, pp. 2022.
Florian Wenzel, Kevin Roth, Bastiaan Veeling, Jakub Swiatkowski, Linh Tran, Stephan Mandt, Jasper Snoek, Tim Salimans, Rodolphe Jenatton, and Sebastian Nowozin. How Good is the Bayes Posterior in Deep Neural Networks Really? In International Conference on Machine Learning, pp. 10248-10259. PMLR, 2020.
Andrew Gordon Wilson and Pavel Izmailov. Bayesian Deep Learning and Probabilistic Perspective of Generalization. Advances in Neural Information Processing Systems, 2020-December, 2020. ISSN 1049- 5258.
Dufan Wu, Kyungsang Kim, and Quanzheng Li. Computationally Efficient Deep Neural Network for Com- puted Tomography Image Reconstruction. Medical physics, 46(11):4733-47776 2019.
Omry Yadan. Hydra A Framework for Elegantly Configuring Complex Applications. Github, 2019. URL
https: //github cww htt////cccbookeeerrr/hh///trrr
Qingsong Yang. Pingkun Yan, Yanbo Zhang, Hengyong Yu, Yongyi Shi, Xuanqin Mou, Mannudecp K Kalra, Yi Zhang, Ling Sun, and Ge Wang. Low-dose CT image denoising using generative adversarial network with Wasserstein distance and perceptual loss. IEFE transactions on medical imaging, (7) 1357, 2018.
Koichiro Yasaka and Osamu Abc. Deep Learning and Artificial Intelligence in Radiology: Current Applica- tions and Future Directions. PLoS medicine, 15(11):e1002707, 2018.
Jin- Yun Yuan and Alfredo Noel lusem. Preconditioned Conjugate Gradient Method for Generalized Least Squares Problems. Journal of Computational and Applied Mathematics, (1(2):287-297, 1996.
Sheheryar Zaidi, Arber Zela, Thomas Elsken, Chris Holmes, Frank Hutter, and Yee Teh. Neural Ensemble Search for Uncertainty Estimation and Dataset Shift. Advances in Neural Information Processing Systems, 34:7898 7911, 2021.
Guangming Zang, Ramzi Idoughi, Rui Li, Peter Wonka, and Wolfgang Heidrich. Intra Self-supervised Learning-based Tomography via Sinogram Synthesis and Prediction. In 2021 IEEE/CVF International Conference on Compuler Vision (ICCV), pp. 1940 1950, 2021.
Ling Zhang, Xiaosong Wang, Dong Yang, Thomas Sanford, Stephanie Harmon, Baris Turkbey, Bradford J. Wood, Holger Roth, Andriy Myronenko, Daguang Xu, and Ziyue Xu. Generaliving deep learning for medical image segmentation to unseen domains via deep stacked transformation. IEEE Transactions on Medical Imaging, 39(7):2531 2540, 2020.
20

---

Published in Transactions on Machine Learning Research (04/2023)
8 Appendices
A Computed Tomography
Diagnostic X-rays constitute the largest man-made source of radiation exposure to the general popula- tion (de Gonzalez & Darby 2004; Picano 2004). In 2010, 5 billion medical imaging studies were performed worldwide, two-thirds of which employed ionizing radiation (Roobottom al. 2010), and the use of radiol- ogy has only grown since (Lin 2010p. Computed tomography (CT), also known computed axial/assisted tomography (CAT), is a noninvasive medical imaging technique frequently used in radiology to generate detailed images of the body and comprises the majority of this radiation exposure (Brenner & Hall 2007). with an estimated 29,000 current or future cancer cases linked to CT scans performed in the United States of America in 2007 alone (De González et al. 2009). Since its original development in the 1970s, CT has become widespread in medical imaging with over 70 million CT scans taken and reported annually in the United States, since 2007 "Smith-Bindunng et al., 2009). There are multiple types of CT scanners, such as spiral CT, electron beam CT, and CT perfusion imaging. In this work, we focus on spiral, also know spinning tube helical, CT.
In spiral CT, illustrated in Fig. the patient lies along the central axis of cylindrical measurement tube, As the scan is performed, an X-ray generator rotates around the patient while moving along the axis of measurement. X-rays are emitted, which pass through the patient and are attenuated at various rates by the different types of tissues in the body, as described in Appendix A.1 After cxiting the body, the allenuated X-rays are measured by X-ray detectors positioned and moving opposite to the X-ray source. These measurements are used create a sinogram, as described in Section A.2 which is not understandable by doctors. This sinogram is then input to reconstruction algorithm, which solves under-determined inverse problem, described in Section A.3 to generate hunaa--nderstaandabl 2D or 3D image of the organ of interest. This image can then be used by the doctor for medical diagnosis.
A.1 X-Ray Attenuation
X-rays produced by CT scanners can interact with matter via the photoclectric effect, the Compton effect, and coherent scattering. Through these interactions, some of the emitted X-ray photons are absorbed scattered when passing through different tissues in the body. The attemuation is described by the Beer-
Adipase Tissue
Bone Cortical
Lens
Mercuric lodine
10a
Soft Tissue
(cm
10"
10'
10-1
0
20
40
60
60
100
120
100
Energy (weV)
Figure Ilustrationn of CT scan- ning device. Reproduced with permission from Brenner & Hall 2007). Copyright Massachusetts Medical Society.
Figure 5: The attenuation coefficient (f) of different tis- sues found in the body, as a function of energy (keV). The plot was generated using X-ray mass attenuation coefficients (f/p) and densities (p) from the NIST Standard Reference Database (Hubbell & Scltzer 2004
Berger ct al. 2004
21

---

Published in Transactions on Machine Learning Research (04/2023)
Lambert Law,
J= Joe where J and Jo are the incident and transmitted X-ray intensities; L is the material thickness; and f is the linear attemuation coefficient of the material,
(14)
f = Ti + T2 T3
(15)
with photoelectric (T1), Compton (72), and coherent scattering (ra) attenuation coefficients. Attenuation coefficients for common materials in the body iodine, bone, water, and soft-tissus are plotted, in Fig.
over the range of incident X-ray energies used in CT imaging. Tt is clear that. the attenuation of X-ray photons can be used to distinguish and, thus, image various tissues in the body (Hsieh 2003)
A.2 CT Projection Measurements
As previously described, the CT scan relies on X-ray generator which rotates around the patient, emitting X-ray photons. In this work, we only consider a restricted case of the spiral CT sctup, in which there is no motion along the patient axis. Instead, we focus on reconstructing singular 2D image cross-sections and assume a parallel-beam geometry, in which photons are emitted and detected with the linear geometry of Figure Ib.
We begin by considering the measurements of single detector, measuring angle ø. Assume that the X-ray generator outputs monoenergetic X-rays of intensity Jo- If the patient were simply homogeneous block of tissue, with length Al and attenuation coefficient f", we could directly apply the Beer-Lambert law (Eq. 14
(16) to solve for the output attemuated X-ray intensity J. In reality, several blocks of tissue will be present in the patient, each with its own attenuation coefficient. However, since the exit X-ray flux from one block of tissue is the entrance X-ray flux to its neighboring block, we can simply apply the Beer-Lamberl law in a cascading fashion over intervals of length Al and attemuation coefficients f2 ........
J == Joe -f
J= Joe fi - - 12 Joe Σ. Si
(17)
As Al + 0, the summation term becomes integration over the length, L, of the patient,
J= S2 (0)rr
(18)
Finally, dividing both sides of the expression by Jo and taking a negative logarithm, we define the projection measurement term,
= -
(19)
As illustrated in Figure D. a CT scanner with a parallel beam geometry contains several detectors side-by- side, collectively measuring a plane of attennated X-ray photons. In this 2D imaging space, the projection measurement becomes a function of delector position, r. Thus, Eq. 19 is re-expressed as the line integral,
J Sa(r) -In (-(r)=-in (1 ( ( - (()rr)ds
(20)
known forward projerd-rooection (FP). It follows from the coordinate system of Figure Ip that, for measure- ments at angle é, point (z, y) within the patient cross-section is projected onto detector position
r' = x cos Φ + yssin ạb
(21)
Combining this with Eq. 20 we derive the Radon transform
(Deans 2007) of the patient cross-section,
Sp(r) --In in (1))
(22)
S f(z,z) 8(x cos + y ó r) drdy,
22

---

Published in Transactions on Machine Learning Research (04/2023)
where 5 is the Dirac delta function and X' y is the set of image pixels (z.y). Sweeping over all the angles, these projective measurements are stacked to form a Radon transform image. As depicted in Fig- ure [ the projection measurements of the blue point across several angles produces a sinusoidal curve. The representation of all the CT scan measurements thus known as a sinogram.
A.3 CT Image Reconstruction Problem
Sinograms are not human- inmaan-nterpretable. They depict the integrated attenuation coefficienss, or projection measurements (r)( of the patient cross-section from several angles (Φ) over all detector positions (r). In- stead, the desired outcome of a CT scan a reconstructed image of the cross-section itself. This corresponds the attenuation cocfficient function, f(r which is the inverse of the Radon transform of Eq. 22 or
f(x,y) 2 /-" uslo cos + y sin Φ) dø,
(23)
where Ug the derivative of the Hilbert transform of Sp(r) (Helgason 1984). The Projection-Slice the- orem (Bracewell 1956) ensures that f can be fully reconstructed with infinite measurement angles, ø. In practice, however, it is not possible to acquire infinite measurements. Typically, reconstruction quality improves with number of measurements, but this increases radiation exposure. In practice, hundreds of mcasurements are performed in a CT scan, but there interest in reducing this number. this work, wo study algorithm performance in the very low measurement data regime, where uncertainty quantficaation over the value of fxx y) becomes especially important. The combination of limited and noisy real-world data renders the reconstruction of the desired image much more complex than simply evaluating the integral of Eq. 23 Leveraging assumptions several statistical models have been developed and used to derive various image reconstruction algorithms, as discussed in the next section.
data-driven insights about the measurements and physics at play,
23

---

Published in Transactions on Machine Learning Research (04/2023)
B Classical Reconstruction Algorithms
In this appendix, we provide brief descriptions of the classical approaches implemented in this work via the TomoPy Astra (Pelt el al., 2016) software package. These algorithms are clinically approved and widely used in medical imaging, serving as a basis of comparison for the methods developed in this work. Note that although our approach used NNs, it was not a data-driven approach, but rather learned image functions from small amounts of data, Thus, we do not discuss data-driven techniques. Further, note that we did not implement the FISTA-TV algorithm, which was also used as classical baseline, but cited results from the ColL work (Sun et al. 2021 where description of the algorithm can be found.
B.1 Filtered Back- Proj-Prooection (FBP)
Filtered back-projection (FBP) Pann et al. 2009) is analytic algorithun which calculates stable, dis- cretized version of the inverse Radon transform. As the name implies, there are two key steps: filtering and back-projoction.
The prrward-rrojctton of Eq. 20 describes how X-rays passing through the object domain create a measure- ment. In back-pree pack-projection (BP), this measurement is integrated back along the X-ray path across the object domain. This is done over all projection angles Φ, using
fer(2, 9) / S ( + sin () do
(24)
to reconstruct the object attenuation coefficient image. As the number of projection angles increases, the image reconstruction improves. However, as shown in Figure G this brok-projection is insufficient to guaran- tee a clear image. While information about the low frequencics of the object are captured in measurements at several view angles, that of high frequencies may only be captured in a few view-angles. Thus, the low frequencies are sampled far more densely than the higher frequencies, resulting a blurry image. This can be corrected by suppressing the lower frequencies with filtering, by applying each projective measurement, (T),, the sequence of Fourier transform (FFT), high-pass filter, and an inverse Fourier transform (iFFT). While several high-pass filters can be used, a popular choice is the Ram-Lak filter, which generates the filtered projective measurement
So(r) - f 1D1=FF(ssl()) [( eeor du:
(25)
where (w) is the Fourier transform of (r) and [e| the frequency response of the filter. Performing back-projections of all the filtered projective measurements,
fFBP(E,V) = f 3 cos o + y ysin Φ) dø,
(26)
results in sharper object attemuation coefficient image. Figure 6 visualizes the difference in reconstrue- tion performance of BP and FBP for increasing measurement view angles, Φ. Although the analytic FBP algorithm is fast and mumerically stable, it suffers from poor resolution-noise tradc-off.
B.2 Algebraic and Iterative Reconstruction
The reconstruction problem can be formulated as a system of linear equations
wf
(27)
where 5 is an m x 1 vector of the m projective measurement values in the sinogram; iis an n 1 vector of the n attenuation coefficient pixel values in the reconstruction image; and W is a, typically sparse, m X n weight matrix representing the contribution of each of the m sinogram values to cach of the n image pixel values. Given the vector S, the goal is solve for f. If W were invertible, would simply be W- However, because is usually much larger than m. the system of equations of Eq. 27 is underconstrained. In algebraic
24

---

Published in Transactions on Machine Learning Research (04/2023)
view
views
15 54
128
05
0.6
Figure 6: Comparison of the reconstruction quality, as function of the number of views, of the BP (top)
and FBP (bottom) algorithms.
reconstruution, this problem is addressed by using iterative algorithms that pose the reconstruction of f as the solution of a constrained optimization problem,
f arg min || 33--Fi Wfll. subject fi 0 Vi.
(28)
Several familics of iterative solvers be used to solve this optimization, such Landweber, Krylov subspaces, and expectation maximization (EM). The key benefit of iterative methods is that prior system knowledge can be integrated, via the cost function and initialization of W. Their down-side is that they are not necessarily stable, may not converge, and are much slower than analytic techniques, such as FBP.
B.3 Simultaneous Iterative Reconstruction Technique (SIRT)
The simultancous iterative reconstruction technique (SIRT) Prryse et al. 1993 Bust & Mitchel] 2008) is a Landweber ilerative method that updales the image reconstruction using all available sinogram projection data, 5, simultaneously. The optimization update at step k is defined as
=(k) f(t) +WW -
(29)
where E mpx is a diagonal matrix containing the inverse row sums, di = (Σ) wiy) and B R is diagonal matrix containing the inverse column sums, ba = EE way) The weighted projection difference, D 1 fTA) ), corresponds to the inverse of the length each X-rays passes through the volume. Shorter rays have a higher contribution, with the weighting required to guarantee convergence. This difference is forward-passed back to the image domain, using the weighted back-projection term, BWT, where it can be used to update the reconstruction. These updates iteratively solve the problem
arg min || - D-WW/ = arg min (5 (S-
(30)
converging to weighted least-squares solution, with weights given by the inverse row sums of W.
B.4 Simultaneous Algebraic Reconstruction Technique (SART)
The algebraic reconstruction technique (ART) GGordon et al. 1970
of the first proposed algebraic iterative algorithms for CT image reconstruction. It is a Landweber technique almost identical to the SIRT algorithm. However, a single projective measurement is used to update the reconstruction image per update step. Generally, the ART algorithm reaches a solution much faster than SIRT, but does not have stable convergence the system of equations is inconsistent, for example due to measurement noise.
The simultancous algebraic reconstruction technique (SART) (Andersen Kak 1984) was proposed 1984, improvement to ART, and also Landweber algebraic iterative algorit hm. Tt combines the reduced
25

---

Published in Transactions on Machine Learning Research (04/2023)
runtimes of ART with the improved convergence of SIRT, by using all the projective measurements from single view angle in each optimization iteration. The update of image vector index i at step is defined as
flat1) -1
An Wij
L+L Σ
Sj S m- Zz Wgi
Wij
(31)
where An << 1 is a, potentially dynamic, relaxation parameter; On is the (n mod N) mcasurement angle of the sinogram, assuming N total measurement angles; and I, is the number of projective measurements taken at each angle. SART typically converges to good reconstruction within a few iterations.
B.5 Conjugate Gradient Least Squares (CGLS)
The conjugate gradient least squares (CGLS) Yuan & lusem 1996) algorithm is Krylov subspace iter- ative method. Since it requires a positive-dfinite definite system matrix, the CT image reconstruction problem is reformulated in terms of the set of normal equations
w WWFf=W'S.
(32)
Due to the positive-definitenes of W W, there cxists a set of conjugate normal vectors Q = ....... Fah: where of WTW,, 0, vi v n). Since Q forms a basis for RP, the image vector f can be rexpressed linear combination of these conjugate normal vectors,
a.q. 1=1
(33)
Thus, solving for f becomes a problem of solving for the conjugate normal basis vector directions, 4, and their corresponding weights, ay, This can be achieved ileratively by expressing the problem quadratic least-squares minimization of the function
(34)
which has gradient VL(F) V²L(f) w W is symmetric positive-definite. The name conjugate gradient least squares comes from the fact that, in each iteration, conjugate basis vector and its weight are found by taking a gradient step in the direction that minimizes the least-squares function, L(f), as
and guarantcod unique minimizer because the Hessian
F(m-1) = =(()) + au
(35)
C = - W (())
(36)
We
To = Ex
(37)
Wg.
ick
(38)
T W Wq.
where En is the residual at step k. Thus, the main difference between SIRT/SART and CGLS is that the search direction in SIRT SIRT/SARD /SART is determined only by the projection difference at that point, while in CGLS the search directions of all the previous iterations are also taken into account. CGLS typically converges much faster than SIRT, but has large memory footprint.
B.6 Expectation Maximization (EM)
The final classical approach to CT image reconstruction that we consider is statistical iterative method known expectation maximization (EM) (Dong) 2007). This technique explicitly encodes prior knowledge
26

---

Published in Transactions on Machine Learning Research (04/2023)
about the X-ray physics at hand. Each projective measurement, Sy is modeled as Poisson distribution,
3
Eー
S S Poisson(AA)
(39)
pj!
where the distribution mean Xy = [E[S is the function
A Eweoh wiyfi
(40)
of the probability wij that an X-ray photon penetrating image pixel was measured at detector location j: and the underlying attenuation coefficient function to reconstruct.
The measurement sinogram is modeled as the likelihood
e
C Wiy -1, wis
II
(41)
Sz!
The EM algorithm computes the maximum likelihood estimate of f.
fMLE =argmax log (p(Sls))
(42)
by alternating between expectation and maximization steps. These can be combined into the update-step
f(a) E, wij E, 1: ((K)
p(++1)
Σ
W
(43)
The EM algorithm is computationally intensive, but guaranteed to converge to a local optimum of the likelihood. Further, although a Poisson distribution was assumed for Sy in this discussion, further knowledge of the detector noise can be casily incorporated into the model.
B.7 Performance of Classical Reconstruction Techniques on Shepp-Logan
Finally, we evaluate the performance of these classical reconstruction algorithms on the Shopp-Logan vali- dation set, depicted in Figure 12 Figure 7 shows average PSNR (for the 5 validation images) as function of view angle, for each of the 5 reconstruction methods. Also shown are the values below which PSNR is usually considered unacceptable (20dB), at which reconstruction is considered lossy (30dB), and above which has high quality (40dB). FBP has the worst performance, which is particularly poor in the low-view regime (< 30 views). The iterative reconstruction algorithms perform better. CGLS has slow convergence to high quality image reconstruction. EM, SIRT, and SART all converge with far fewer views, achieving lossy compression with only ~0 views. EM levels out and requires around 180 views to achieve high-quality reconstruction. SIRT and SART have near identical performance, passing the high-quality reconstruction threshold with only 75 views and SIRT performing slightly better for larger view numbers.
Table 3: Classical reconstruction PSNR, averaged across the five validation set images, in the 5-, 20-, and 180-view cases. The best achieved PSNR for each view-# is bolded.
# Views
FBP
CGLS
EM
SART
SIRT
7.68
16.38
21.39
21.12
21.12
20
17.35
21.85
30.22
31.98
31.97
180
36.74
38.6
40.46
42.51
42.76
Table reports the PSNR values obtained for 5, 20, and 180 views. EM is the best performer for views, SART for 20, and SIRT for 180. However, while the performance of the three algorithms is similar for 5.
27

---

Published in Transactions on Machine Learning Research (04/2023)
35
30
25
SIRT
SART
EM
CGL5
5-
125
Views
Figure PSNR as a function of number of views for the classical reconstruction algorithms.
SART and SIRT perform better than EM for larger number of views. In the low-view regime, roughly an order of magnitude required to achieve a 10dB improvement in average PSNR. Figure [ shows a reconstructed image for each of these algorithm-view combinations. With views the reconstruction algorithms are able capture low-frequency object structure, but the image would be useful for medical diagnosis. With 20 views is clear that the algorithms already capluring high-frequency components of the object, but the images have many artifacts. By 180 views, the reconstructed images nearly identical to the ground truth images of Figure 12 with any discrepancios in PSNR due mostly to minor image reconstruction artifacts imprecisions.
FBP
CGLS
EM
SART
SIRT
5 views
3.0
0.8
0.6
20 viewo
180 views
Figure Reconstructions, generated with varying view angles, by the classical reconstruction algorithms the Shepp-Logan validation set.
28

---

Published in Transactions on Machine Learning Research (04/2023)
C
Metrics and Uncertainty
C.1 PSNR & SNR
PSNR is defined as
mfx
PSNR((";;) 10 10
(44)
MSE. ())
where MSE(f", [X
(x) V
2,
while SNR is defined
|lf 1
SNR(J",)) -20 10
(45)
where f denotes the ground truth image, f the noisy/reconstructer image, and || ll2 the 2--norm. SNR is strictly less than PSNR, with higher SNR and PSNR corresponding to better image reconstruction. the absence of any noise, I* and I are identical, SNR and PSNR infinite. For lossy images, PSNR is typically between 30-50dB, with values over 40dB considered very good, and values below 20dB considered unacceptable Bull 2014
C.2 Types of Uncertainty
Bayesian modcling can address two distinct types of uncertainty: alcatoric and epistemic DDrr Kiureghian & Ditlevsen 2009; Kendall & Gal 2017). Alcatoric uncertainty is due to measurement noise, such X-ray detector noise. This type of uncertainty cannot. be reduced, even if more measurements are taken, since it is inherent to the measurement. To see why, think of rolling an unbiased die. Irrespective of how many times you roll the die, you will always be uncertain of the outcome the next roll, since each outcome has ā probability. On the other hand, epistemic model uncertainty accounts for uncertainty in the model parameters. This type of uncertainty can be reduced with more measurement data. To see why, imagine model that aims to predict the outcome of biased die roll, with no prior information about the bias. As more data is taken, the variance in the model parameters decreases, and the model output distribution better approximates the true biased die distribution. Even in the presence aleatoric uncertainty, this work primarily focuses on quantifying epistemic/moole uncertainty. Namely, we consider how well the model reconstructs the ground truth attenuation coefficient image from the sinogram data.
C.3 Calibration and Coverage
Calibration is a metric that assesses a model's ability predict the probabilities of its outcomes, gauging the reliability of the model's confidence in its prodictions. For example, model performing class predictions is considered calibrated if it assigns class 50% probability and that class actually appears 50% of the time in prediction. For further information calibration of class prediction models, we refer the reader to (Guo ltt al. 2017 Nixon et al. 2019). Since this work focuses on regression models, the remaining discussion is centered on calibrated regression (Klleshov et al. 2018).
Let F be the cumulative distribution function (CDF) of model predictions f(, y), that seek to approximate ground truth image f E F, where F denotes the functional space of possible images. Letting fz = f(x v) We denote the corresponding quantile function
(Ø) inf P F(f)).
(46)
where F-1 performs the mapping F-1, [0, + F and 5 is a confidence interval. For calibrated regression, ground truth pixel f(x y) should fall in a, say, 90% confidence interval 90% of the time. Thus, the regression model calibrated for confidence interval p
T Σ ==1
lim x X
1faa < FF)
P VD E [0,
(47)
29

---

Published in Transactions on Machine Learning Research (04/2023)
Ground Truth
Mean
Variance
MSE
Coverage
NLL
Figure 9: The ground truth image is plotted alongside different metrics for assessing the reconstructed predicted distribution generated by ULNR model. From left right, we show the ground truth, predicted mean image, predicted variance image, mean squared error, coverage quantile of each pixel, and negative log-likelihood of each pixel. Note that PSNR is calculated using the ground truth and predicted mean image. In real medical setting, the ground truth is unknown, the doctor would be given the predicted mean image and the predicted variance image could be provided as supplementary information help the doctor reach diagnosis. Further note that white regions in the coverage image denote that the ground truth pixel value did not fall in the range of the predicted distribution.
as the number of pixel samples approaches infinity, X + co. If f denotes the ground truth value for i.i.d. random pixel (z, y) E X, a sufficient condition for calibrated regression is
pl23 S (T) = D. v p 1].
(48)
Since practical dataset sizes are finite, preventing perfect calibration, different metrics have been developed to assess empirical model calibration.
Reliability diagrams serve visual representation of model calibration, plotting expected sample accuracy as function of average model confidence. Ideally these would be continuous plots, but, in practice, samples are binned into M bins according to their prediction confidence. Let Bm be the set of indices, i, of samples with prediction confidence, P; in the interval I = (" M1 The expected accuracy (acc) and confidence (conf) are approximations to the terms of Eq. 48 namely
1
p(f* S Fx (i)) acc(Bm)
Σ
1f F (0))
(49)
[Bm] i.
1
n conf(Bm) Bm
Di.
(50)
The calibration error (CE) is the discrepancy
CE = p < (x) - aco (co(BB))coo([BB3 CE( (.)
(51)
It can be measured on reliability diagram as the difference between the expected accuracy curve and the idcal acc(B,.) = conf(B,.) line. The expected calibration crror (ECE) quantifics the calibration crror of the full distribution as
ECE(J', FTPK(";Fx'00
1
Σ acc (B.)) con((Bm)) |-
(52)
M
The model is considered calibrated if ECE(x, f) = 0.
In practice, modifications were made to the previously described theory of reliability curves and ECE. You may notice in Figure I0 that, instead of plotting accuracy and confidence, instead plot analogous target coverage and achieved coverage. Typically, coverage value, P, refers to quantile of data points lying within +5% of the median (50% quantile). Specifically, in our setup, we use different. uncertainty quantification methods (BBB, MCD, and DE) to sample N different model weights for our INR, each set of weights corresponding to a different model output. Given that each output corresponds to an image, for each pixel, (z. v), we have distribution of N predicted values, Fy (x,y). y). Ideally, the median of the pixel distribution would be equivalent to the ground truth pixel value, f f* u)- However, this is unrealistic to expect in
30

---

Published in Transactions on Machine Learning Research (04/2023)
Reliability Curve
6 Selection Curve
With
Without
0.8
Achiouad Coverage
0.0
0.25
0.20
0.4
0.15
0.2
11100
0.05
0.0
10-14 10-12
10-10
10
10-
Target Coverage
Figure 10: Both plots were made with an MCD UINR model trained on 20 views, achieving PSNR of 19. Left) A plot the model reliability curves, with the grey dashed line indicating a perfectly calibrated model. The blue curve the empirical reliability curve of the model when a small term added symmetrically to the target coverage, in order to slightly widen the quantile ranges. Although this 6 term has nearly negligible magnitude, it significantly improves the model reliability curve, illustrated by the orange curve of reliability without the added 6 term. Right) The added d term was not chosen arbitrarily, but selected to minimize ECE.
practice. Thus, we check whether the ground truth pixel lies within the predicted pixel distribution quantile, Qn specified by coverage value, p,
Qs (FN(2. (x, Qs0+ (Fv( (50+g(Fw(r,)))
(53)
If a model is perfectly calibrated, 5% of reconstructed pixels distributions will contain the ground truth pixel in their p% quantile, corresponding to the grey dashed line Figure I0 Thus, model confidence can seen as a pre-selected quantile for each pixel (target coverage), p, while accuracy the percentage of pixel distributions containing the ground truth that quantile (achieved coverage),
1 X
ΣΣ JEX DEy
{ Qo0- Q000-1 (Fx(2,,)) S (z, S Qs0+ (Fx(a.)))
(54)
The ECE is thus implemented as,
RCE(F"FN))
1 IP
Σ AC(J",FNI -/, PEP
(55)
where P is a finite set of percentages evenly spaced in [0, 1], separated by percentage interval i <<< 1. The fifth image Figure g plots the smallest quantile of each pixel containing the corresponding ground truth pixel, for an example UINR reconstruction with N = 50. Note that white regions indicate that the ground truth value docs not fall within the minimum and maximum predicted pixel values.
There is one final modification made in implementing the reliability curves, in order to effectively assess model calibration. Since the final layer of all the NNs used for the INR have sigmoid activation, ensuring that the model output is in the range (0, 1). However, the sigmoid function only approaches 0 and 1 in the infinite limit, meaning that in practice our model will never output 0 or exactly. However, our images contain a large percentage of pixels with exactly value, especially for noiseless artificial data, which in the context of medical imaging is regions containing air and no tissue. This problematic for calibration, since all of predicted pixel values will be near-zero, but will not actually contain the ground truth value of 0. This is iltestrated by the orange reliability curve in Figure 10 for which only 40% of pixels contain the ground
31

---

Published in Transactions on Machine Learning Research (04/2023)
truth in their full range of predicted pixel values, for an MCD model trained on 20 views with N = 50. Our proposed solution to this issue is slightly widening the quantile range by adding a negligible ö term. Thus, for coverage value p, we now check if the ground truth pixel lies in,
2s0 (FN(zz)) - ó S (x, S 250+ (FN(1,))) + +6,
(56)
this case, if our predicted pixel values are slightly larger than 0, the offset can widen the quantile range include 0 enabling these pixels to contribute to the achieved calibration (this also applies to pixels with exact value of 1). The improvement in using delta offsct is illustrated by the blue reliability curve in Figure TO which is much closer to the ideal grey dashed line than the orange curve with 8. It should be noted that the value of ó is not assigned arbitrarily, but instead optimized to minimize overall ECE. For too small a 5, the quantiles will not be widened sulficiently to capture ground truth 0 pixels. However, for too large of 5, the quantiles will be widened too much, reducing overall calibration, as achieved coverage is much higher than target coverage for low coverage values. Thus, ECE as a function of 8 expected have unique minima, illustrated by the example in Figure I0
where < <<
C.4 Assessing Model Quality
Section 4.1 describes how PSNR and SNR quantify image reconstruction quality, coverage metrics (such as ECE) gauge the uncertainty calibration, and NLL encapsulates both. this work, we aim to optimize both reconstruction and calibration quality, meaning the best metric would, naively, be NLL. However, there is often trade-off between calibration and prediction quality. Specfically, NN overfitting to NLL manifests in probabilistic error rather than prediction error (Guo el al. 2017). Furthermore, while this work focuses on uncertainty quantification of INRs for medical imaging, little prior work has addressed this problem. Most cxisting techniques only quantify reconstruction PSNR and SNR. Hence, for fair comparison, we assess and optimize our models primarily according to PSNR and SNR. However, for similarly performing models, we use NLL and FCE as secondary selectors for the best model. Note that initial attempts at optimizing models according to coverage metrics resulted in preference for the lowest capacity models possible. This indicates that optimal performance according to coverage favors blurry image reconstruction, with as little certainty possible in the final image.
32

---

Published in Transactions on Machine Learning Research (04/2023)
D Bayesian Deep-Learning Methods and Implementations
Building off of the high-level descriptions of the different BDL methods discussed in Section more detailed algorithms descriptions and implementation specifies.
B we provide
D.1 Mayes-by-Baccpaee
A. popular method for variational approxination lo exact Bayesian updates is Bayes-by-Backrrop (BBB) (Blundell et a 2015). BBB aims to find the optimal parameters & of an approximate distribu- tion on the NN weights, q( also known as the variational posterior. This is achieved by maximizing the variational free energy/evident lower bound (ELBO),
CS, (b = [g (log p(S))) - KL (0l() (())
(57)
with respect to the variational parameters sb. The ELBO can be optimized with respect to & using stochastic gradients estimated by Monte Carlo,
())( & V [logp(S))) + log log ((0())),
(58)
where 0 ~ ((11)) is a sample drawn from the variational posterior, and the gradient taken through 0 using the reparameterization trick (Rezende et al. 2014).
In this work, the BBB variational posterior is treated a Gaussian distribution, Wi (w) The elements of By comprise diagonal covariance matrix, meaning weights assumed to be uncorreaaeed A Gaussian prior, p(0) = o²), with tunable is used to initialize the network. Training the network requires computing a forward-pass and backward-pass. Although the network is parameterized by a distribution of weights, in each forward pass a single sample is drawn from the variational posterior and propagated through the network to perform updates. A re- paraamttteriitooon trick (Kingmaa & Welling 2014), in which the sample e transformed by the function Pe + ag 3 c, usod ensure a gradient can be calculatod for backpropagation. Finally, to aid learning, is common to modily the ELBO
(gg, R
(og [[S0) E IKL (0))) |1 (()),
(59)
where E> is an added hyperparameter, known as the Kullback-Leibler (KL) factor. This beneficial for training because it puts greater emphasis the training data in the loss, through the (how))/logp(())) In the context of medical imaging with INRs, this re- reigess the importance of obtaining a Radon transform of network outputs close to the sinogram measurement data.
D.2 Monte Carlo Dropout
Another popular approach is Monte Carlo dropout (MCD) (Gal & Ghahramani 2016 There, the authors argue that optimizing a NN regularized with dropout (Srivastava et al.) 2014] applied to every layer can be interpreted as variational approximation for a deep GP. The first two moments of the corresponding variational posterior can be approximated using Monte Carlo, with N samples from NNs sampled with dropout. The predictive mean thus calculated as,
N Σ f(0m)))
(fo) ~
(60)
N
which is equivalent to averaging the results of N stochastic forward passes through the network. Details of the MCD implementation provided in the "Implementation Details' of main paper Section
D.3
Hamiltonian Monte Carlo
A final mcans of BNN inference which we explore in this work, is Hamiltonian Monte Carlo (HMC) (Duane
et al. 1987 Neal et al. 2011: Betancourt] 2017). Originally proposed for calculations in lattice quantum
33

---

Published in Transactions on Machine Learning Research (04/2023)
Typical Set
d0
p(Ols)
p(O|S)de
0
mosic
Gull
Figure 11: An illustration of concentration of measure, in which the expectation, p(0)S) of the BNN's high-dimensional weight posterior is concentrated in the typical set. This can be attributed to the fact that, while most of the distribution density p(0[S) is concentrated about the distribution modes, the volume do is concentrated at the tails. Unlike other MCMC methods, HMC efficiently explores and samples from the entire distribution typical set. (Figure inspired by (Betancourt 2017
chromodynamies (Duane al. 1987). HMC an instance of the Metropolis Hatropolis-Hastingss algorithm (Chib & Greenberg 1995) for Markov Chain Monte Carlo (MCMC) (iikss al., 1995). In the context of quantifying NN uncertainty, HMC can be used to obtaín sequence of random samples that converge in distribution to samples from the BNN posterior distribution, p([[S) (Izmailov et al., 2021 For the high- -ise-iimensonnal distributions associated with BNNs, the density S) ) is concentrated at the distribution mode, while the volume d0 is comcentrated at the distribution tails. The resulting distribution expectation - a product p(0]S)dd of distribution volume and density concent a nearly-singular neighborhood, known as the typical set, as illustrated in Figure [1] Traditional Metropolis Hastings MCMC, using Gaussian random walk proposal distribution, typically fails to explore the full typical set of these distributions. HMC, however, leverages the physics of Hamiltonian dynamics via time-reversible and prremmeirreeeiingggg integrator, to simulate and propose points scattered around the typical set. This significantly improves exploration of the full distribution and decreases the correlation between consecutive samples, reducing the total number of required MCMC samples. In this work, HMC was implemented via the NumPyro Python package (Phan et al. 2019). using leapfrog integrator and No-U-Turn sampler (Hoffman et al. 2014). For a more detailed description of the HMC algorithm, in the UncertaINR context, we refer the reader Appendix E
D.4 Deep Ensembles
Alternatively, predictive uncertainty can be quantified by aggregating the outputs of several NN "base learn- ers" trained for the same task from different initializations, a method known as deep ensembles (DEs) (Lak- shminarayanan et al.) 2017). Averaging predictions over multiple NNs consistent with the training data leads better predictive performance, enables uncertainty quantification, makes DEs robust model misspocifi- cation, and presents a strong baseline in -f--iiiiiitiiitionnn detection (Ovadia et al., 2019). Ensembling can also be applied on top of other methods to further improve performance. In principle, more base learners results in better performance, but, in practice, training large ensembles is computationally expensive and diminishing returns are observed after ~10 base learners. In this work, UncertaINR leverages ensembles of M Monte Carlo dropout base learners, such that Eqs. and TO are updated to,
N/M Σ
for(z.
Σ m
fem. (x.y)
(61)
M N/M
1
VDE(D.y)
Σ
(Jom.n fD)(
N
34

---

Published in Transactions on Machine Learning Research (04/2023)
The relationship between Bayesian inference and DEs is an active area of research in the BDL community.
Wilson & Tzmailov (2020) argue that DFs provide more compelling approximation to the true posterior than many standard BDL approaches, whilst others have adapted DEs to provide a Bayesian interpretation
(Ciosek et a 2019 D Angeelo & Fortuin 2021 Pearce et al. 2020 He et al. (2020) characterize how DEs
relate to posterior inference in the limit of infinite NN width.
Typically; DEs induce randomness by training the same network several times with randomized initializations and data order. However, recent work (Zaidi et al.) 2021 has shown that ensembling over architectures can outperform the more common single-architecture DEs for uncertainty estimation. For our large-scale hyperparameter study (on Shepp-Logan phantom data), in which thousands of BBB and MCD UncertaINR base learners were trained, we were able to easily implement architecture-ensemblel DEs, by selecting the best-performing UncertaINRs as base learners. However, for the high-resohution AAPM-Mayo data, in which UncertaINR training times were extended significantly, we found too computationally demanding to hypertune multiple architectures to ensemble. Thus, for the final results presented, DEs are created by ensembling the same network trained with randomized weight initialization.
D.5 Hardware and Software Notes
The experiments presented in this paper were computationally intensive, requiring hundreds of compute hours on parallelized GPUs. Runtimes for each of the methods developed and implemented in this work on the AAPM dataset are presented in Table Non-HMC methods were run on a cluster of GPU nodes consisting of 8 GPUs each, containing mixture of GTX 1080, GTX 1080Ti, and GeForce RTX 2080 Ti cards. HMC methods (*) were run on two Titan RTX cards, which were faster and had double the memory. Note that the HMC models were further intialized to a pre-trained check-point, which helped the models converge much faster. The time required to obtain this pre-trained checkpoint is not reflected in the HMC runtimes. Furthermore, while overall runtimes of the ensemble methods (?) appear long, each of the base models could be trained in parallel.
Most experiments were run on a GPU cluster consisting of four GPU nodes with 8 GPUs each, consisting of a mixture of GTX 1080, GTX 1080Ti, and GeForce RIX 2080 Ti cards. Experiments denoted with a (*) were run on faster Titan RTX cards, with increased memory. (t) is used to denote ensemble experiments, which in practice were run parallel to achieve faster runtimes.
Table 4: Computation Times:
RECONSTRUCTION METROD 60- VIEW RUNTIME 120- ViEw RUNTIME COMPUTE TYPE
GOP-TV
< 10 MIN
10 MNN
GTX FTX/GFFOORC TITAN RIX* GTX /OO/GE
HMC GOP-TV
25- MIN*
45+ MIx* *
INR
~22 IRs
~2.75 uns
DE.2 CINR
44 HRS!
~5.5 Has!
GTX//EFOOR /GEFORRF
DE-5 CINR
~10 Hast
~11 HRs!
GTX GTXX/GF /GEFORRF
DE-10 UINR
~0 hast
~27. has!
GTX/ /GEFORCE
MCD UINR
2225 mRs
~2.8 mRs
GTX CTRRCEEORCE
DE.2 MCD UINR
<5 HRS!
~5.6 Has!
GTT//EFFORCE
DE-5 MCD UINR
~ 11.25 hast
11 unst
GTX/GE
DE-10 MCD UINR
22.5 HRs!
~28 ues!
GTX/GE
IMC UINR
8+ IRs*
11.5+ mss*
TITAN RTX*
Overall, we do not forsee computation as a limitation of UncertalNR. In fact, this work demonstrates that MCD (one of the least. computationally intensive means of UQ) is highly effective for INR UQ, while adding minimal overhead to the INR evaluation time. Furthermore, as mentioned in Section 2.2 recent work on INR optimization has found ways to reduce INR evaluation times. These approaches could be implemented directly within the UncertaINR framework to reduce training time, without affecting the network results or uncertainty quantification. This, however, would neither add any novelty or change the fundamental claims of this paper, which are about UQ and not computational speed.
The project codebase was developed in Python, mostly using Pytorch (Paskke al. 2019). Hydra Yadan
2019), and Weights & Biases (Biewald 2020) to implement the NN functionality and Blitz (Esposito. 2020 for BNN functionality. For HMC, we used the No- No-U-Turn-Sampler (Hoffman et al.
2014 sampling scheme
in NumPyro (Phan et al. 2019), which is based in JAX (Bradbury et al. 2018).
35

---

Published in Transactions on Machine Learning Research (04/2023)
E HMC Sampling Algorithm for UncertalNR
Our goal in using HMC with Une is to sample weight parameters, (8.,)..., B(0), from the BNN weight posterior, p(0[S). This problem is reformulated in terms of physics-inspired dynamics. These dynam- ies are governed by Hamiltonian
1 PT M M
H1O, = (() U 2
Po.
(62)
where 0 are the 'position' terms, Po are the 'momentum' terms, U(0) - In p(]S) is the system potential energy, P M PP is the system kinetic energy, and M is symmetric positive definite mass matrix. HMC starts by initializing parameters, g00) ~ (O, +), by sampling from the Gaussian distribution of prior precision At HMC iteration n, the parameters are initialized to g(a) (0) g(m) while the momentum is initialized by sampling from the normal distribution (0))(0) ~ (o M) of variance defined by the mass matrix. The leapfrog ilerative algorithm then used to simulate system dynamics for time LâI, where L is the number of leapfrog steps and At the step size. In each step, the leapfrog algorithm alternates between momentum and position updates, using
At p( (1) ((m) 1+
At VU(0))0=00 (=))0(=))))()))
(63)
2
2
0 (t + At) (t) + AtM (())
At
++
(64)
p nd (t +
At) (t)
P(a) (+
At
At
VU VU( (00)===)))(0a)))
(65)
so as to solve Hamilton's 's equations
d0 0H
(66)
di 0P,
dPg 07l dt 00"
(67)
Since the leapfrog iterative algorithm is a discretized numerical approximation to the true integral, the limit
Al + 0 would be needed to solve Hamilton's equations exactly. To correct for bad proposals from the leapfrog method, an HMC Metropolis- Hastiiss acceptance ratio is defined as
(0), = min 1, expl ( (LAt)) (LAr)) (0), (0))
(68)
where H. is the Hamiltonian defined in Eq. 62 In result, the parameter sample returned by iteration which is also the parameter initialization of iteration n + is
(Lin) with probability (O)) otherwise
0(m) (LAt))
( (l (0), glm) (LAt) (LDt)
(69)
This process is repeated for T" HMC iterations. Since the prior weight initialization does not usually lie within the typical set of the distribution, the samples initially output by HMC are poor indicators of the typical set region. To address this problem, a burn-in period of initial iterations is defined and all burn-in samples are discarded after the algorithm complete.
36

---

Published in Transactions on Machine Learning Research (04/2023)
F Preliminary Ablation Study
Our preliminary ablation study presents a large-scale study of uncertainty quantification for INRs. Specif- ically, we test the relative ability UncertalNR (using MCD, BBB, and/or DEs) to reconstruct artificial noiseless CT brain images. From this study, we present guiding principles for effective UncertaINR hyper- parameter selection and compare to traditional CT reconstruction techniques.
F.1 Dataset
Given the long INR. training times required to reconstruct large, high-frequency images (Fridovich-Keil al. 2022), opted for dataset of simple images, enabling large-scale hyperparameter sweeps. Specifically, the Shepp-Logan phantom (Shepp & Logan 1974) approach was used to generate (256 x 256 pixel) artificial brain images, with corresponding measurement sinograms generated via the Radon transform. No noise was added to the mcasurement sinograms. In all, 10 ground truth images, depicted in Figure 12 and 20 corresponding sinograms were generated: 5 validation and 5 test set sinograms each for the and 20-view (Ø) cases.
Image #1
Image #22
Image #3
Image #4
Image #5
Validation Set
0.8
Test Set
0.2
Figure 12: The ground truth images used in tuning and assessing our preliminary Shepp-Logan hyper- paramter exploration. The five validation set images were used to optimized model hyperparameters, while test set images were used to assess the finalized models.
F.2 Baselines
this study, we compared UncertaINR to the classical medical image reconstruction algorithms FBP, CGLS, EM, SART, and SIRT described Section 2.1.1 and Appendix B implemented using the TomoPy Astra wrapper (Pelt al.) (2016). detailed analysis of the classical reconstruction methods on the Shepp- Logan validation set is presented in Appendix B.7
F.3 Tuneable INR Model Parameters
There are several degrees of freedom in designing INR, including its size, embeddings, activation functions, and optimizer. These design choices are critical in determining model performance, but there is little theoretical understanding of how to best select most of these model parameters. In this section, we lay out the different parameters we considered in designing our INRs and provide any known insights as to how they affect model performance. These insights informed the hyperparameter sweeps described in the following section.
F.3.1 Width and Depth
The size of an NN determined by both its width (number of nodes per layer) and depth (number of layers). Universal approximation theorems (Hornik al. 1989) have been derived in both the arbitrary-
width (Cybenko) 1989. Hornik 1991) and arbitrary-depth (Hanin 2019 Kidger & Lyons 2020 Lu et al.
37

---

Published in Transactions on Machine Learning Research (04/2023)
2017 cases, demonstrating that NNs are theoretically guaranteed universal function approximators in the infinite limit. In practice, however, neural networks have finite width and depth. Recent work has empirically demonstrated and theoretically suggested that, in this regime, increased-depth networks generally perform better than increased-width networks (Lu et al. 2017 7 Raghu et al. 2017). It is also known that, while neural networks are overparametrized relative to the amount of training data, this overparamettriation is key for al., 2019). However, for INRs specifically, it has been shown that relatively small networks can typically be used to learn decent functional image encodings (Dupont et al. 2021 Thus, we tend to sweep over smaller widths and depths than standard deep-learning networks.
their generalization ability (Jacot et al., 2021 Neyshabur
F.3.2 Fourier Feature Mappings
Random Fourier features (RFF) were first introduced in 2007 by (Rahimi & Recht, 2007 a means of accelerating kernel methods. The key idea is to map the input data to a randomized low-dimensional feature space, while maintaining the kernel of the original data. Given input I E IR, the RFF mapping takes the form
TRFF (i) = [cos(27 2n B (2x BT))²
(70)
where B is m matrix, with each entry sampled from (N(0, (2)) The standard deviation, Gg is tuneable hyperparameter, but remains static after initialization i.c. it is not modified with NN weights during the MLP training. There exist other types of Fourier feature mappings, such positional encodings, in which
(F) = [..., cos (2 #), 1)),snn (2n(22 ") "".","""
(71)
for = O,..., m
In 2018, it was theoretically demonstrated that NNs can be approxinated by kernel regression via the neural tangent kernel (NTK) (Jacot et al. (021)). Using this intuition, in 2020, it was argued that applying simple Fourier feature mapping to inpul data enables MLPs to learn high-dimensional functions rapidly, even in low- dimensional problem domains (Tancik et al. 2020), making the technique particularly well-suited for INRs. In fact, positional encodings have been shown to have key importance in the success of NeRF (Mildenhal] et al., 2020) and Fourier feature mappings have been shown to boost the performance of the CoIL network for medical image reconstruction (Sun et al. 2021 J. In this work, all networks apply an RFF mapping, TAFF
to the input data. The standard deviation, Ro, is tuned among other hyperparameters.
F.3.3 Activation Functions
Activation functions are key to the success of neural networks, transforming what would otherwise be simple lincar systems into complex, non-linear universal function representers. We performed hyperparameter sweeps with five activations widely used in the MLP and INR literature ReLU, SiLU, Sine, Soft and Tanh. We now briefly review these activation functions, as well as their use in deep learning.
The rectified lincar unit (RcLU), plotted in blue in Figure 13 was introduced as carly as the 1960s for visual feature extraction in hicrarchical NNs (Hara et al. 2015 The ReLU is defined as
ReLU(x) max (0.
(72)
returning its input if greater than zero and otherwise returning zero. Despite its hard non-linearity at zero, non-differentiabilit at zero, and vanishing gradient challenge {Lu et al. 2020), the ReL.U was shown in 2011 to enable better training than previously used activation functions, such as Sigmoid and Tanh, by inducing sparse representations (Glorot et al. 2011). As of 2017, the ReLU was the most popular activation function for deep NNs (Ranachandran et al. 2017).
The
linear unit (SiLU), plotted in orange in Figure 13 is specific instance of the Swish ac- livation function family and was proposed in 2017 as continuous, 'undershooting" version of the ReLU RRa- machandran et al. 2017). The Swish family, parameterized by B, is defined as
Swishs(x) x o(3x) 1
(73)
38

---

ReLU, SiLU, Sine, SoftPlus, and Tanh tested in our Published in Transactions on Machine Learning Research (04/2023)
2.5
ReLU
SiLU
2.0
Sine
SoftPlus
1,5
Tanh
Activation(a)
1.0
0.5
0.0
0.5
-10
Figure 13: The five different activation functions preliminary hypertuning experiment.
where (x) is the sigmoid function. By setting 3 to different values in [0, oo). Swish non-linearly interpolates smooth functions between the linear function and ReLU. In 2017, Swish was empirically shown to outperform ReLU, a result thcoretically attributed to its bounded, smooth, and non-monotonic nature (Ramachandran et al. 2017). More recently, Swish has been shown to outperform both ReLU and Sine the conlext of CT image reconstruction via Automatic Integration (AutoInt) (Lindell al. 2021 The SILU is the specific instance of Swish where
SiLU())=x-o(() = 1te=
(74)
The Sine activation function, plotted in green in Figure 13 the sinusoid
Sinew = sin(wo w)).
(75)
In the 2020 SIREN paper (Sitzmann et al. 2020 INRs with sinusoidal activation functions and random Fourier features were empirically demonstrated to outperform ReLU-based INRs. Theoretically, it argued that these periodic activations are better suited to capturing naturally complex signals and their derivatives. However, the performance of these activations depends strongly on the choice of frequency, wo. which needs to be tuned.
The SoftPlus activation function, plotted in red Figure 13 has continuous and differentiable form
Softplus(z) = In + c*).
(76)
It was introduced in 2001 (Dugas et al., 2001 as the primitive the sigmoid function. It primarily used as smooth approximation the ReLU activation and to constrain to positive outputs, since Softplus(z) e (0, oo).
The hyperbolic tangent Tanh, plotted in purple in Figure 13 has form
E" -
Tanh(x) e² +e-
(77)
and is both differentiable and monotonic. It has form similar to the sigmoid function, with Tanh(z) (2r) - 1, but lies in the range (-1,1) instead of (0,1), meaning it docs not constrain to positive values. Before the ReLU became popular, the sigmoid and Tanh were two of the most common activation functions. Tanh was easier to train and typically outperformed the sigmoid as an activation function. However, because these sigmoidal activation functions saturate for large inputs, their derivatives vanish for these inputs, leading slow convergence of learning algorithms. This has motivated the increased use of ReLU-like activation functions (Goodfellow et al. 2016), which ameliorate the vanishing derivative problem.
39

---

Published in Transactions on Machine Learning Research (04/2023)
F.4 Experimental Design
The goal of this hyperparameter study was to understand the relative performance of UncertaLNR with BBB, MCD, and DEs. In all these cases, well-tuned hyperparameters were needed to achieve decent model reconstruction accuracy and uncertainty calibration. Large hyperparaneter sweeps were used to find the optimal parameters for BBB and MCD. The best performing MCD UncertaINRs were used as base learners for the DEs.
F.4.1 BBB & MCD
For both BBB and MCD, ML. design choices had a large effect on INR reconstruction performance. Carefully designed hyperparameter sweeps were thus run to strategically search the MLP parameter space for four different settings: (1) MCD 5-view, (2) MCD 20-view, (3) BBB 5-view, and (4) BBB 20-view.
The model selection process began with coarse grid search to efficiently prune across the wide range of possible parameter combinations. Specifically, we considered model activation type, depth, width, RFF embedding frequency Po, and dropout probability. Among these, activation type was the only categorical parameter, using the five activation types plotted in Figure [3] Tanh, SoftPlus, Sine, SiLU, and ReLU, For the remaining paramcters, this initial coarse grid search was usod get a sense of orders of magnitude, for the sake of computational feasibility. We swept over model depths of 3, 6, and 9; widths of 16, 64, 256, and 1024; and RFF Dg's 's of 1, 5, 10, and 15. Three values- 0.2, 0.5, and 0.8 were considered for the final parameter, dropout probability, which is specific to MCD. For BBB, we instead swept over the Gaussian prior standard deviation (values 10, 100, and 1000 and KL factor (values 1c-10, 1c-5, and 1 For these coarse grid searches, all networks were trained using the Adam optimizer with no weight decay and the default learning rate of 3c-4. For each set of parameters, three individual INRs were trained, one for each of the three validation images, Image #1-#3, shown in Figure LE All reported metrics are averaged across the three test images, in an effort to ensure model generalization and prevent overfitting to a particular image. For both the 5- and 20-view experiments, 2,160 (2,512) models were trained and tested for the MCD (BBB) coarse grid sweeps. This resulted in total of 9,344 models trained and tested during these initial grid searches. Since the performance metrics are calculated from distribution of predictions, sampled according to the uncertainty method, all hyperparameter sweeps used 50 prediction samples to enable uncertainty quantification, mcan output prediction, and metric calculation.
For now, we conclude our discussion of methodology, with the second hyperparameter sweep a fine Bayesian search used to generate the final models. This Bayesian sweep leveraged a reduced search space, informed by the first coarse grid search. It was found that Bayesian sweeps do not perform well with categorical variables, so independent sweeps of 200 runs were performed for each of the three best performing activation functions from the grid search. Since only 600 models were trained in total in each uncertainty-viev setting, all five validation images of Figure 12 were used as the validation set, to improve model generalization ability, Further, AdamW was used to optimize the models, with a weight decay hyperparameter added to the sweep. In the case of MCD with 20-views, three sweeps were performed, one for each of the three activation functions: Sine, SILU, and Tanh. A log uniform distribution, in the range 1c-16 to 1c-1, was swept for the weight docay, while uniform distributions U(min, max; q), where q is the discrete interval, were generated and swept. over for the remaining numerical parameters: depth U(2, 12, 1), width e U2200, 1000, 100), Go U33, 15, 1), and p(dropout) E 10.1,, 0.6, 0.1). The top performing model according to PSNR, across all three Bayesian sweeps, was selected as the final model.
F.4.2 Deep Ensembles
Given the robust and computationally intensive nature of DEs, we did not perform large-scale hyperparam- eter sweeps, as was the case for BBB and MCD. DEs combine the outputs of multiple base learner models
We originally swept over BBB standard deviation (values 0.2 0.5, and 0.8), which are more par with theoretical expectations. However, found that increasing prior standard deviation significantly improved final model performance. GGieen the added BBB uncertainty hyperparameter, we reducod relative number of search values for the remaining sweep parameters.
7 AAgain, for the sake computationally efficiency, only validation Tmages #1 and #2 were used for BBB.
40

---

Published in Transactions on Machine Learning Research (04/2023)
to improve uncertainty calibration. Assuming each base learner makes a reasonable prediction, even if not optimized, adding more base learners should only maintain or improve uncertainty calibration. In order to create a DE of size M (DE-M), the top-M performing models identified by the MCD hyperparameter sweeps were used as base learners. If N total samples were desired for uncertainty quantification, cach of the MCD baselearners was sampled repeatedly to generate M predictions. These predictions were pooled logether to create a sample of size N, from which uncertainty was quantified, the mean prediction generated, and model metrics calculated. In order remain consistent with the BBB and MCD experiments, we used N 50.
N
F.5
Uncertainty Quantification Method Performance Analysis
F.5.1
MCD Hyperparameter Analysis
For MCD hyperparamcter sweeps, metrics were averaged across the INR model reconstruction of the first three validation set images of Figure 12 However, we also recorded the PSNR of the INRs trained on each individual image. Box plots for the individual distributions PSNR for validation Images #1, #2, and #3 are shown in Figure 14 Idcally, PSNR should be consistent across the three images. This roughly the case (barring a few outlier points) for models using the Tanh, SoftPlus, Sine, and Silu activation functions. Notice, however, that in all cases the distríbution is broadest for Image #3. This effect is exacerbated for models using the ReLU activation function, for which PSNR of Image #3 ranges all the way from approximately OdB to nearly the maximum achieved PSNR, in both the 5- and 20-view cases. This indicates that ReLU in particular, but all the all activation functions to some extent, struggle to capture features of Image #3.
Rel U
SIU
Sine
Softplus
Tash
Iede lmuge PSNR
25-
Figure 14: Reconstruction PSNR for Images #1-3 of Figure 14 for MCD INRs with different activations
and different numbers of views.
To understand what is unique about Image #3, the image reconstructions of the best overall performing model are shown, for each activation function and 20-views, in Fig. [5 It is apparent that Sine and SiLU produce smoother images, while Tanh, Softplus, and ReLU produce spottier images. Except for SoftPlus, all activation functions manage lo capture low-frequency image information and strong edges. However, all activations struggle to capture the small, low-intensity ellipses in the center of Image #3. Overall, it is clear that, irrespective of activation function, 20-views are insufficient to robustly capture fine CT image details. This individual image analysis also suggests that the Sine activation performs the best for MCD, achieving the highest overall PSNR.
Figure 16 shows boxplots of the average PSNR distributions for MCD models trained in the 5- and 20-view cases. Each column contains all the models trained with one of the five activation functions, while each row shows how the PSNR distribution changes as function of hyperparameter depth, width, probability of dropout, or RFF frequency Gg- Ideally, PSNR would be consistently large across hyperparameter values, indicating that the architecture is robust and does not require much tuning. In practice, however, we find that the activation functions are either consistent or high- nigh-performing, but not both. As previously mentioned, Sine achieves the best overall PSNR. However, it is also the Icast consistent activation function, with its highest performing models typically being outliers (indicated by diamonds in Figure 16). Softplus, on the other extreme, is very consistent across hyperparameter values, but performs consistently poorly. Tanh, Silu, and ReLU have less extreme variations. Their top models perform slightly worse than the best Sine models, but they perform much more consistently across hyperparameter values (ReLU is a bit inconsistent in width and probability of dropout). This suggests that the Sine network is potentially the best reconstruction
41

---

Published in Transactions on Machine Learning Research (04/2023)
SoftPlus
Image #11
Image
0.4
Image
0.0
Figure 15: Best image reconstructions obtained with cach activation function, for 20-view MCD.
network, but significant tuning (in terms of hyperparameter search) elfort may be needed to achieve that solution. For practitioners inclined to perform less tuning, the Tanh and SiLU networks may be preferred solution, due to their robustness and competitive top performance.
For ReLU, Tanh, SILU, and Sine, it should also be noted that the PSNR distributions behave similarly in the 5- and 20-view cases (with 5 views performing consistently worse than 20 views, as expected) for all hyperparametens except RFF 24- This is consistent with recent results (Tancik et al. 2020) suggesting that RFF embeddings enable NNs to learn higher frequency image information. In the 5-view case, where the available data insufficient for the INR to confidently learn high-frequency image features, performance is poor for increasing Ωg. In the 20-view case, the increased data enables the network learn higher frequency image components. However, because higher-frequency Qo is required to ensure the network learn those frequencies, best performance tends to occur for larger Ωn- This effect can also be observed in Figure 17 For 5 views, the roconstruction is blurry for Ωo = but the network starts to learn smaller ellipses for 24 = 5. By Ωo 10, however, the network is trying to learn higher-frequencies than the data cannot specify, resulting in artifacts, which are exacerbated for Qg = 30. the 20-view case, a similar trend of reduced blurriness and increasingly sharper images can be seen between f4 an Go 10. However, the reconstructed images have much more recognizable details and less artifacts than those obtained with 5 views. It. is only for Ωo 30 that artifacts start. to appear.
actually
42

---

Published in Transactions on Machine Learning Research (04/2023)
ReLU
SIUU
Sine
Softplus
Tash
Aog PSWR
View
Dayoth
Tlesth
RaLU
SIUU
Sine
Softsluu
30-
Aeg PSNR
Viess
10-
254
1024
16
64
256
1024
16
1004
256
1004
16
256
2004
Wiibh
aki
Wilth
ReLU
SILU
Sine
Softplus
Tanh
Avg PSRR
10-
P(Drepout)
Droppus)
P(Deopour))
P(Dropoott)
P(Dropoat)
RaLU
SIU
Sine
Softplus
Tash
25
Avg PSMR
10 30
30
10
RSF 12
RFF
RFF
RFF 5l
RFF 04
Figure 16: Boxplots of the average PSNR of MCD models trained in the coarse grid search hyperparameter sweep, for both 5 and 20 views. Each column corresponds to different activation function and cach row to sweep over one of the remaining hyperparameters- depth, width, probability of dropout, and RFF frequency Pg. Individual diamond points are outliers.
43

---

Published in Transactions on Machine Learning Research (04/2023)
MCDropout 5-Views
ReLU
SiLU
Sine
SoftPlus
Tanh
(No = 10
On = 30
MCDropout Views
ReLU
SiLU
Sine
SoftPlus
Tanh
Do
86 33
(3 = 10
Dg 300
Figure 17: MCD image reconstruction, in both the 5- and 20-view cases, for each activation function and RFF frequency Go valuc. Note that in the 5-view case, So 5 enables the network to learn low-frequency image leatures, without many artifacts. Smaller Po causes the network to produce overly simple output, whereas larger Qo induces high-frequu figh-freeuency artifacts. In the 20-view case, similar observations are made, but the optimal 2n = 10. In this case, the reconstruction has higher frequencics without significant artifacts, for most activation functions.
44

---

Published in Transactions on Machine Learning Research (04/2023)
F.6 Comparing the effects of RFF frequency for DEs and MCD
In Section 4 saw that MCD is effective approach to obtain calibrated UQ in INRs, compared to DEs. This is surprising observation, particularly in relation to the typical belief, in BDL settings (e.g. image classification), that deep ensembles are a very strong or statc-of-the-art basclinc (Ovadia et al. 2019
While we do not have a full justification for this observation, we suspect that the underperformance of DFs (relative to MCD), in the context of UncertaINR, is related the capacity that each individual ensemble member has. In particular, we observed that the RFF frequency No is a very important hyperparameter. This frequency is related to the 'capacity' of the model, in the sense that higher S4 implies ability to learn higher frequency details of the image, as shown in Fig. 10 of Tancik et al. (2020).
In Fig. 18 we plot the effect of RFF frequency Ro for different numbers of sinogram views (5 and 20), for both MCD and DEs. For 20-view Shepp-Logan data, we observe that an ensemble (blue) achieves roughly the same PSNR at both Qg = and f2, (top right). However, the NLL at Qg = 40 is significantly better (lower) than the lower capacity Q4 (bottom right). This indicates that the models with higher RFF frequencies generate more diverse, but still accurate, predictions and hence obtain better uncertainty estimates. On the other hand, MCD introduces accurate predictions through different dropout soeds at set dropout rate p. Thus, dropout combined with RFF encodings allows the individual models to have sufficient capacity to produce diverse yel accurate predictions. In contrast, deep ensembles rely solely on the RFF encodings to regulate the model capacity (and thus, diversity in predictions). This is also corroborated in Fig. 18 by the fact that MCD achieves better NLL (on Shepp-Logan phantoms) at lower S2 (i.e. lower capacity) than deep ensembles.
alternative effective way of encouraging diverse yet
A more formal understanding of the settings in which deep ensembles are effective, as well as interactions with MCD, is important future work both for UncertaINR and BDL generally.
views=5
D. views=20
24
P5NR
15.0
12.5
-2.
-30
x
>0
10 20
40
omega_im omega_init_aaye
omega Laagggannitt_aaa layer
Figure 18: The effect of changing RFF frequency Go for PSNR and NLL with MC dropout and Deep Ensembles.
45

---

Published in Transactions on Machine Learning Research (04/2023)
F.6.1 BBB Hyperparameter Analysis
The BBB model selection analysis is presented a similar fashion to that of MCD. The main differences MCD are that, in the BBB grid search, metrics are averaged over only the first two validation images; the width and RFF search spaces reduced; and the uncertainty parameters prior standard deviation (not probability of dropout). Figure I9 shows boxplots of average PSNR a function of different hyperparameter values. Unlike MCD, model performance is extremely consistent across activation functions for every typerparameter, except for width. We note that Tanh has some variation for RFF Ωo following trends similar to those discussed in the case of MCD. Overall, SiLU performs the best, with ReLU and Tanh following closely behind. Interestingly, Sine performs the worst, failing to reach even 20db. However, greater performance consistency across hyperparameter configurations comes the price of weaker top-performing models, which achieve lower PSNR values than those of MCD. Because the image sets not identical, these comparisons across approaches should be taken with some reservation.
the KT. factor and Gaussian
To understand why there is so much variation in BBB performance where the average PSNR obtained for each width is plotted as a function of prior standard deviation. It can be seen that the PSNR values are extremely consistent for each network width, across prior standard deviations. From a Bayesian perspective, the fact that the prior does not affect inference suggests that the latter is dominated by the model likelihood. However, mean performance decreases as function of model width, indicating that the model becomes increasingly misspecified for larger widths. Given the Gaussian assumptions made in the variational inference specifications of BBB, this suggests that the true posterior distribution becomes less Gaussian as network width increases, and the variational approximation deteriorates. When combined with the low performance of BBB relative to other uncertainty quantification methods, reported in Table 1 this indicates that BBB may not be well suited for uncertainty quantffication of INRs in the medical imaging context.
function of width, consider Figure 20
Furthermore, the overall BBB ablation results appear to mimic Coker et al. (022)'s observation of mean- field BNN performance degradation with increased width. In all, BBB's poor performance is unsurprising and well-established in the BDL literature (Ovadia et al. 2019) (here BBB=SVI).
Table 5: Top 10 performing MCD models and their performances, for the 5-view case (Top) and 20-view case (Bottom). Models are ranked by PSNR, but NLL and ECE are also reported.
RANK
ACTIVATION
DEPTH
WIDTH
RFF O
PD
DECAY
PSNR
NLL
ECE
-1.437
0.122
2
SINE
600
01.4
0.157
25.79
-1.799
0.008
3
SINE
3
600
8
0.7
0.366
25.76
-1.579
0.009
4
SINE
3
700
2
0.4
4.462-4
25.75
-1.533
0.117
9.36E-4
-1.390
0.011
6
SINF
500
0.7
0.077
25.63
2.622
0.161
7
SINE
700
3
0.6
0.012
25.62
-1.149
0.123
8
SINE
3
500
5
0.7
0.068
25.62
-1.643
0.007
9
SILU
0.2
28682-7
25.62
0.219
0.389
10
SINE
500
0.7
0.170
-1.79
0.083
ACTIVATION
DEPTH
WIDTH
RFF f
PD
W. DECAY
PSNR
NLL
ECE
SINE
3
100
9
0.4
2.06g-5
33.74
0.701
0.134
SINE
4
300
12
0.4
2.63E-7
33.41
1.076
0.141
SINE
500
0.5
0.006
33.37
-0.502
0.137
SINE
3.74F-6
33.29
SINE
400
10
0.2
5,85E-6
33.28
4.634
0.132
SINE
500
8
0.2
1.117-44
33.24
2.622
0.161
SINE
400
9
0.5
0.001
33.21
-0.798
0.143
SINE
500
2.536-4
33.20
SINE
500
7.876-7
33.18
1.973
0.163
SINE
500
0.5
1.56E-5
33.17
-1.257
0.114
46

---

Published in Transactions on Machine Learning Research (04/2023)
ReLU
SILU
Sine
Softpls
Tanh
Vinwe
6 Dcath
6 Disthh
6 Dipti
Dipti
Dijoc
ReLUU
SILU
Sine
Softlle
Tanh
Dog
20
255 Width
2:55 Wilthh
Width
Widch
ReLU
SILU
Sine
Softples
Tanh
12
Viows
5
10
3
10
30
10 KFFT Ob
30
30 RFF Oo
10 RFF f
30
IeFF Rie
RITT Ska
Rel
SIU
Sine
Softples
Tanh
20
1o-05
10-05 Sine
20-05 Softples
ReLU
SIU
Tanh
Viowa
10
120
1003
10
100
1000
10
100
1000
10
100
1000
10
100
1000
Price Std. Dev.
Prior Std. Dex.
Prior Std. Dex.
Prior Stad. Dec.
Pr Std. Dex.
Figure 19: Boxplots of the average PSNR of BBB models trained in the coarse grid search hyperparameter sweep, for both 5 and 20 views. Each column corresponds to different activation function and each row to a sweep over one of the remaining hyperparameters depth, width, RFF Go, KL factor &, and prior standard deviation o.
F.6.2 DE Performance Analysis
As described in Appendix DEs of M base learners created by combining the top-M performing NNs, according to average PSNR. As reported in Table [ the best MCD model outperforms the best BBB model significantly, with at least a 2dB increase in PSNR, as well as reduced NLL and ECE, for both the 5- and 20-view cases. Thus, we ensembled the top MCD models produced by the second fine Bayesian hyperparaneter sweeps of Section [.6.1] The parameterizations and performance the 10 best performing models for both the and 20-view cases are listed in Table 5 Note that, in both cases, the model architectures vary greatly across models. While the Sine activation function is fairly consistent, the remaining parameters vary greatly. For example, in the 5-view case, model depths range from to widths
47

---

Published in Transactions on Machine Learning Research (04/2023)
Prior Sigma 10
Prior Signa
100
Prior Sigma 1000
20
Aug PSAR
15-
Views
5-
32
256
1024
256
1024
256
1024
Width
Width
Width
Figure 20: Boxplots of average PSNR of BBB models of varying width for each value of prior standard deviation o, for both and 20-view cases.
Deep Ensemble 5- 5-View
25.49
2,500
m
PSNR
26.30
0:07
MSE
hw
0.455
26.00
2,400
26. 15
-2.4
0.05
2.575
-2.
0
20
40
10
30
50
10
90
20
30
10
50
# Base Learners
Base Learners
# Base Learners
Base Learners
Deep Ensemble 20-Vies Viee
0.100
0.50
54.2
PSNR
NL
MSE
0.50
0.050
-0.75
# Base Learners
# Base Learners
Base Learners
Base Learners
Figure 21: PSNR, NLL, ECE, and MSE plotted as a function of the number of base-learners used in DEs, for both the 5- and 20-view cases. Note that each added base learner performs slightly worse in terms of image reconstruction quality than the network preceding
from 300 800, RFF Do from 2 to 8, and probability of dropout (PD) from 0.2 to 0.7, all with a variety of weight decays. These variations increase base learner diversity well beyond different weight values.
DE combines varied base learners to improve uncertainty calibration. In principle, all base learners achieved the same PSNR, model performance should only increase (or plateau) and variance should only decrease (or plateau) as more base-learners are added. In our case, however, each new added base learner had a slightly lower PSNR on the validation set. To verify how this affected model performance and calibration, we generated plots of each metric as function of # of DE base learners. As shown in Figure both PSNR and MSE improve significantly as the first base learners are added the ensemble, but begin worsen for larger ensembles. Considering that models of worse PSNR are being added with each increase in of base learners, it is unsurprising that performance eventually degrades. However, ensembling never reduces performance below that of using the single best model. NLL and ECE are more sensitive to the number of base learners. NLL demonstrates the overall best. performance gain as a function of baselearners. ECE, however, does not change consistently with DE size, with large ensembles sometimes even harming performance relative to the single best model. In all, is clear that ensembling improves model performance and calibration. However, larger ensembles have no gains over smaller ensembles and are far more computationally expensive. Thus, for the final results, presented in Table only ensembles of sizes 2, 5, and 10 are considered.
48

---

Published in Transactions on Machine Learning Research (04/2023)
Ensembling
DE-5
1.0
PSNR:
0.1
0.001
030
ECE: 0 07076
Wile
1.0
00.
Wii
Figure 22: Image reconstruction and calibration performance of the final DE models on test set Image #3 for 5-views (top) and 20-views (bottom). Different columns show the results of different ensemble sizes, ranging from 1 to 10. Top row shows the reconstrueted image, middle row the pixelwise coverage, and bottom row the reliability curve.
Figure 22 illustrates image reconstruction performance for the different ensemble types on test set Image #3. In the 5-view case, DFs achieve impressive performance improvements, with PSNR increase of over 1.5dB for DE-5 and an ECE reduction of 0.011 for DE-2. In the 20-view case the baseline is much better performing. Hence, although there are gains in PSNR, these are not very noticeable in the reconstructed image. However, the improvements in calibration are larger, with an ECE drop of 0.2 for DE-2 and clear improvement the image reliability curve.
F.7 Final Results Conclusions
The primary goal of this study was to understand the relative performance of different uncertainty quantifi- cation techniques for UncertaINR, on the Shepp-Logan dataset. Table T in the main text, presents metrics assessing the pest-perrorming MCD, DE-2 MCD, DE-5 MCD, and DE- 10 MCD UncertalNRs. These methods consistently outperformed the classical reconstruction techniques in terms of image quality, while producing reasonably well calibrated uncertainty estimates. BBB was found be the worst performing uncertainty quantification approach, generally producing the poorest calibrated uncertainty estimates and worse image
49

---

Published in Transactions on Machine Learning Research (04/2023)
Reisbiity
Wall
DE-5
Figure 23: Validation results of all approaches for 5-view Shepp-Logan Logan data. From left to right: mean image reconstruction, variance, MSE, coverage, and reliability diagram.
Mean
Variance
MSF
Coverage
Relibblit
B888
MCD
06:2
DE-5
DE-10
Figure 24: Validation results of approaches for 20-view Shepp-Logan data. From left to right: mean image reconstruction, variance, MSE, coverage, and reliability diagram.
50

---

Published in Transactions on Machine Learning Research (04/2023)
Table Ablation Study Differences: Test set results from Table presented again, in terms of differences from the average metric value for each image. Specifically, the table reports the average and standard error of these differences across the 5 test images for each metric.
RECONSTRUCTION
V-View TEST SET DIPFERENCES
20-View TEST SET DIFFERENCES
TYPE
PSNR (1)
NLL (L)
ECE (L)
PSNR (1)
NLL (1)
ECE (U)
FBP
-14882-- 0,239
13.200 0,168
CGLS
-5.397 1. 0.201
-8006 1. 0.272
EM
-0.136 1: 0.093
0.199 0.364
SART
-0.270 0.167
SIRT
-0.270 1: 0.167
0.261 1,533
BEB UINR
2.504 =: 0.378
1,524
0.383
0,105
0.000
-0.749 :: 0.152
0.782 = 2.137
0.013 + 0.008
MCD UINR
1.437 = 0.244
-0.186 + 0.094
-0.015 0.003
1.170 = 0.218
0.329 = 0.532
0.007 + 0.012
DE-2 MCD UINR
1.173 = 0.267
-0.389
0.063
-0.028
0.006
1.529 = 0.276
-0.153 = 0.580
-0.005 = 0.011
DE-5 MCD UINR
4.866 0.402
0.070
-0.032
-0.05
DE-10 MCD UINR
4.655 1 0.303
-0.583 1 0.262
-0.030
0.006
4.957 0.144
0.554 1 0.519
00,000 1 0.011
reconstruction than classical techniques in the 20-view regime. MCD consistently outperformed classical approaches and was generally better calibrated than BBB. Ensembling over MCD base learners, however, was the most successful approach, outperforming the best classical approach by ~4dB the 5-view case and ~3dB the 20-view case, as well as achieving the lowest overall NL. and ECE values.
Given the varying difficulty of images in the Shepp-Logan test set, we did not believe that reporting standard deviations in Table would accurately reflect reconstruction variability. Instead, Table the average and standard error over the difference from the mean metric value for each image. Calculating the difference from mcan for each image reduces the effect of variability in image difficulty. Notice that the differences are significant with respect to the standard error across distinct methods, although not as siguificant across ensemble sizes for the MCD UINRs.
One further remark regarding Table 1 is that, although the classical reconstruction procedures did not use the validation set images as validation (since no hyperparameters were tuned), image reconstruction quality still deteriorated in the test set. This indicates that the test set data is actually more challenging than the validation set. Given that UncertalNR performance did not significantly decline the test set, we have strong reason to believe that none of the UncertaINR approaches over-fitted to the validation set.
Figures 23 and 24 visually illustrate the difference in the uncertainty quantification of the different methods, for the 5- and 20-view cases respectively. Beginning with the 5-view case, the mean predicted image was fairly blurry for all methods, only capturing low- frefrreuency image components and exhibiting high uncertainty surrounding edges. Furthermore, note that all reliability curves are very well calibrated in this 5-view case (after 5-selection adjustment); except for BBB. Similar trends are present in the 20-view case, but it is visually harder to distinguish differences in the image output due to the higher quality of all reconstructions.
51

---

Published in Transactions on Machine Learning Research (04/2023)
G UncertalNR AAPM Performance Assessment
G.1 Dataset
In order to assess the performance of UncertaINR in realistic setting, used data from the American Association of Physicists in Medicine (AAPM) and Mayo Climic 2016 Low-Dose CT grand challenge (McCol- lough et al. 2017)- Specifically, we use the reconstructed images used as a test set the ColT. work (Sun et al. 2021). shown in Figure 25 so as to directly compare to their results. These ground truth images were reconstructed from real CT scan measurement data.
Figure 25: The 8 AAPM-Mayo test set images used to assess UncertaINR performance.
In order make the dataset more realistic and compare the results presented in CoIL, noise was added the image sinograms before being imput to the models. This artificial data generation pipeline illustrated in Figure 26 More specifically, uniformly distributed Gaussian white noise added to the sinogram, to achieve a desired SNR relative to the original, noise-less sinogram. In this work, chose to present results for a noise level achieving sinograms of 40dB SNR.
Ground Truth Image
Sinogram
Reconstracted Image
(s)
(b)
(c)
Noisy Sinogram
Noisy Image
(d)
Figure 26: A flowchart of the artificial data generation pipeline used in this work. (a) The Radon transform is used to generate sinogram, corresponding to measurement data, from the ground truth data. (b) In the noiseless case, this sinogram would be directly used to train our model. Note that even though the sinogram is noiseless, the reconstruction is not perfect. (c) In a more realistic scenario, Gaussian white noise added the sinogram.
(d) This noisy sinogram data is then fed into the model, which produces a reconstructed image of the ground truth, of lower quality that produced in the noiseless case.
G.2 Final Model Hyperparameters
Given the increased computational costs in running UncertaINR on the AAPM-Mayo dataset, it was not feasible to perform large-scale hyperparameter sweeps, like those performed in the ablation study presented Appendix(F) However, coarse scarches were performed for each hyperparameter and new training approaches were used order achieve the competitively performing models presented in Table 2 For reprodueíbility, provide the final hyperparameters and training procedures.
52

---

Published in Transactions on Machine Learning Research (04/2023)
Beyond training insights learned from the hyperparameter study, further improvements were made to Un- certaINR training in order to achieve competitive reconstruction accuracy on the higher-resolutin, higher- frequency, and noisy MayM data. Specifically, images were zero-padded to ensure that all image content was contained in the projections when calculating the Radon transform. All UncertaINR were opti- mized using Adam (not AdamW, to eliminate the computational costs of luning the weight decay parameter) and we found that longer training times were needed (on the order of 15,000 epochs). If that many training epochs cannot be used for some reason, we found that stochastic weight averaging (Izmailov et al. 2018) can be used to improve reconstruction accuracy by few decibel for UncertaINRs trained with few epochs. With the increased number of training iterations, the Sine activation proved too unstable and we found SiLU to be the best performing activation function, with Tanh a close second. Noting the significance of RFFs from the ablation study, we found it crucial that the width of the RFF layer be al least as large as the padded image width. Finally, before adding a regularization term to the UncertalNR loss, our models were not competitive with statc-of-the-art reconstruction tochniques. While adding an isotropic TV regularization,
Tiso(f)
E (f(x+ 5(x+1,0) 1,)) f(), (f(2. 1) (zz, E.EEXy
boosted performance by few decibel, the anisotropic approximation of the TV regularizer,
TAN
E x.EEXxy
If(x+ 1,9) f(..v)) (f(2, f(x, v)l.
achieved better reconstruction accuracy, especially in the low-measurement (60-view) regime. (at the slight cost of uncertainty calibration). The relative performance of these two regularizers relative to no TV reg- ularizer are presented for both GOP and MCD UINR in Table 7 While we do not have justification for why the anisotropic approximation performs better than isotropic for reconstruction accuracy, we believe that regularization/ prior exploration would be interesting future work.
Table 7: The affect different regularizers on image reconstruction performance.
RECONSTRUCTION МETHОN
REGULARIZATION TYPE
60- VIEW SNR
120-VrEw SNR
NoNR
18.33
18.85
GOP
ISOTROPIC
23.62
25.60
ANISOTROPIC
25.97
27.40
NonE
25.75
27.35
MCD UINR
ISOTROPIC
25.78
28.06
ANISOTROPIC
27.38
28.65
Given all these training insights, the hyperparameters of the top-performing (MCD and HMC) UINRs. reported in Table 2 are presented in Table g
Table 8: Hyperparameters of the top-performing MCD UINRs, HMC UINRs, and INRs reported Table Additional HMC-specific hyperparameters, such as burn-in periods, are provided in Appendix G.2.1
Mopen
# VIEWS
ACT.
FUNc.
DEPTH
WIDTH
RFF fo
REC TyPe
REC COFFF
D(ROOOOU))
# EPoCHS
280
0.05
15,000
MCD UINR
SILU
5
250
35
AN1SO
0.03
0.2
15,000
HMC UINR
60
SILU
4
200
48
ANISO
0.03
N/A
N/A
INR
120
SILU
5
280
60
ANISO
0.05
0
15,000
MCD UINR
SILU
280
40
ANISO
0.008
0.2
15,000
HMC UINR
120
SIT.U
4
200
60
ANISO
0.03
N/A
N/A
G.2.1 HMC-specific hyperparameters
As mentioned in Section 3 all our HMC experiments used the NUTS Hoffman et al. 2014 implementation
in NumPyro (Phan et al. 2019).
All HMC runs for 1000 samples and had burn-in period of 500
53

---

Published in Transactions on Machine Learning Research (04/2023)
samples, with thinning factor of 5, that every fifth post burn-in sample was used at evaluation time. We initialized the model parameters from trained model for both GOP and UTNRs (corresponding to models in the GOP and INR rows of Table respectively) to enable faster mixing. For NUTS specific hyperparameters, we set the max_tree treetteeeeee depth to and target a Metropolis- Hastings acceptance rate of 0.8. During the burn-in period, NUTS adaptively sets the step-size and (diagonal) mass matrix hyperparameters which are crucial to the effectiveness of HMC; more details can be found in Hoffman al. (2014).
HMC-GOP hyperparameters For 60 and 120 views we set the TV regularisation strength to 0.5 and 0.3 respectively. Both values were tuned on small grid of hyperparameter values.
HMC-UNIR hyperparameters We set the parameter variance, 72 in Eq. to V7xwidth" with ? 0.2 and
0.15 respectively for 60 and 120 views. 7, and the regularisation coefficients in Table @ were tuned on a small grid of values due to the high computational cost of HMC. Due to the non-convex nature of NN parameter space, we aggregated the samples from two independent chain runs (from two independently trained MAP initializations) for HMC-UINR, the maximum number of chains that fit into memory 24GB VRAM Titan RTX GPU.
G.3 Results Analysis
Given the varying difficulty of images in the AAPM test set, we did not believe that reporting standard deviations in Table 2 would accurately reflect variability. Instead, presents the average and standard error over the difference from the mean metric value for each image. Calculating the difference from mean for each image reduces the effect of variability in image difficulty. Notice that, generally speaking, our conclusions from Table 2
regarding the different methods (e.g. MCD provides more calibrated UQ than ensembling and our LNR-based methods outperform their classical counterparts) are statistically significant relative to standard error across different images.
Table 9: AAPM Differences: AAPM results from Table 2 are presented again, in terms of differences from the average metric value for each image. Specifically, the table reports the average and standard error of these differences across the 8 test images for each metric.
RECONSTRUCTION
60-VIEWS
120 N0-ViEws
(1)
ECE
(1)
NLL (1)
ECE (1)
EM
-9.7
9.34 0.33
CGLS
+3.06+0..22
-3.55 + 0.26
SIRT
-3.16
0..0
SART
-2.50:1:.11
-3.72 0.10
GOP-TV
1:011
1.01 :: 0.07
IMC GVP-TV
1.011
-4.70
-0.004 0.011
1,33 0.12
-4.00 0.81
-0.07 0.00
DE-2 UINR
3:25++0.000
22.45 4 3.41
0.118 0.004
3.34 =: 0.10
17:02 +: 2.30
0.114 # 0.00M
DE-5 UINR
3.26
6:32 1 0.93
0.070 : 0.006
3.34 0.00
7.57 8: 1.29
0.075 0.007
478 1.900
0.010
MCD UINR
3:34::0.11
-6.50 0.72
-0.229 0.005
2.14 0.13
-158 0.61
-0.DD è: 00006
DE-2 MCD UINR
3.41
-0.68 0.77
-0.44
2 0.004
3.21 11113
-5.3 0.64
-0,02 0.006
DE-5 MCD UINR
3.44++0.11
-0.00
0.005
-5.11
0.07
DE-10 MCD UINR
3/12::0.10
-3.79 0.8
-0.01
0.004
5.25 0.13
-5.32
-0.055 0.006
HMC UINR
3.00
-6.07 1 0.90
-0.00
0.010
3,01 0.11
-0.023 1 0.008
G.4 Observed Cold Posterior Type Effect
We observe Table that there a decrease in SNR performance of HMC GOP-TV relative to its determin- istic counterpart GOP-TV (25.10 vs 25.97 for 60 views), the latter corresponding the maximum-a posttrrio (MAP) point estimate for the posterior, Eq. 6 from which seck to sample.
This evokes similarities with the recently observed "cold-posterior" effect (CPE) in BDL. (Wezel et al. 2020). In this setting, one considers a tempered posterior Pf((S) x exp( (ap(- -V(f),, where the Bayes' posterior corresponds to temperature T = and "cold" posteriors (T < 1) place more weight parameter regions
54

---

Published in Transactions on Machine Learning Research (04/2023)
of high posterior probability. In the cold limit T - 0+: the tempered posterior becomes exactly the MAP
point estimate.
For context, the tempered version of Eq. 6 for GOP yields:
[
(S. - A.f)* - -
PT((S) & exp
(78)
2T
The CPE is the observation that colder postcriors (T < 1) outperforms the standard Bayes' posterior (T 1)
in terms of generalization, and understanding its causes has been a topic of recent study in BDL Wenzel
et al.
2020
Adlam et al. 2020 Aitchison
2020 Fortuin et al. 2022 Wilson & Izmailov
2020 Izmailov
et al.
2021
Noci et al. 2021 Nabarro et al. 2022
Inspired by this line of work, well as our observation that HMC GOP-TV underperforms MAP GOP-TV (point estimate) in terms of SNR in Table we studied the effect of tempering posteriors in GOP with HMC in 17 We find a similar effect to the cold-posterior whereby SNR performance is improved at cold temper- atures, and even allows tempered GOP-HMC to outperform its deterministic MAP counterpart. However, we also observe that the colder temperatures are detrimental to the metries which consider UQ: ECF. (which is purely about UQ) and NLL (which considers both UQ and reconstruction accuracy), presumably because the tempered sample only samples around the MAP estimate with little diversity in samples.
There are three commonly presumed causes for the cold-posterior effect (Noci et al. 2021 data-curation: data-angmentation; and prior mispecification. Given that our observation model (Eq. 5) is constructed by design, and (hat we use noisy data exactly matching our likelihood model, we rule out data-curation (proposed by Aitchison (2020) who argue that the CPE is caused by the fact that standard DL datasets, e.g. CIFAR-10, are well-curated and that standard likelihoods like cross-cntropy do not account for this) as the cause for our observed effect in 27 Likewise, our CT reconstruction setting is very different standard DI. supervised learning and there natural analogue for data-augmenlation. This leaves prior misspecification, in the form of the TV-reuularization term in Eq. 78 a potential cause for our observed cold-posterior like effect in CT reconstruction.
We note that the grid-of-pixels model uses a simple grid-based pixel-by-pixel dimension array of parameters without any mention of NNs, and enjoys a convex log-posterior compared to the more complex non-convex parameter loss landscapes that exist in DB) In this sense, the best of our knowledge 27 represents the first suggestion that a cold- ood-poottrror effect can occur outside of NN-based models.
5NR difference MAP GOP-TV (1)
NLL (L)
ECE
MAP GOP-TV
0,250
150
Metric value
125
0.225
-0.2
100
0,200
-0.4-
75
0.175
-0.6-
50
0,150
25
0.125
-1.0-
0.100
10
-0
10
10-4
10-1
10
10
10-1
10a
Temperature
Temperature
Temperature
Figure 27: A cold cold-posterior like effect observed for GOP-TV with HMC for 60-v AAPM.
G.5 UncertalNR AAPM Test Set Outputs
In our main figure of results, Figure 3p, we present the model outputs and metries for only 1 out of the 8 A APP test. set images, illustrated in Figure 25 We conclude the paper by presenting the figures for all
55

---

Published in Transactions on Machine Learning Research (04/2023)
test set images. Note that the observed trends and analysis, presented in main paper Section for the remaining, following test set image results.
4.3
hold
Like
OBa
LiN
IBCC GOP
DC LIE
NCD LIVE
DCC MED UNNE
OE-DOMD MCD UIEE
IBC UNR
Toget
Toger Cooraee
Toger Coergge
Corrae
Tapt Gorigge
Tapt Cretag
Tagsl Somaa
Tage Couag
Taga Goraage
Tage Cart
DC:00
UNR
URB
HSX GOP
MCD LBN
DB-S UN
DE MEG LTBE
HOX LEW
Tage
Teass Conrae
56

---

Published in Transactions on Machine Learning Research (04/2023)
Seceen www.weeeeeeee.. Mode
Abolia Sno Pooonaratteeiii Man
Konite Rowe
Varanc
Connerr
Crerag
HNC GDP
DE-3 UPB
NCO UMB
DeS MDD UNT
DE-00 MCE UNt
HMC UIM
Amadun Ent
Camtugg
57


Reviews:
[{"review_id": "22CbnYJ2Yn", "reviewer": "Reviewer_RRpV", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "\nStrengths:\n\n- Interesting task. Despite the technical contribution is limited, assessing the calibration performance of image reconstruction is an interesting and important topic. Calibration of discriminative models is gaining popularity in discriminative models. Nevertheless, to my knowledge, improving the uncertainty estimation on image reconstruction remains unexplored. \n\n- Literature review is properly conducted. Existing works in relevant topics are properly discussed, which better motivates the goal of this work, i.e., addressing the calibration performance of uncertainty quantification of implicit neural representation models in image reconstruction. \n\n- The empirical results obtained by the different proposed methods outperform classical reconstruction methods. As stated by the authors, deep learning based methods (Table 2) outperform the proposed approach but they require more data. Nevertheless, I have some concerns on these results that I will discuss in the weaknesses section.\n\n\nWeaknesses:\n\n- I think that extending the proposed method to other modalities would better highlight its generability. Furthermore, having only one artificial and one real benchmark could be suboptimal to showcase the superiority of the presented approach.\n\n- The work of Reed et al (Dynamic reconstruction from limited views with implicit neural presentation and parametric motion models) should be included in the experimental section, as to my understanding they also reconstruct CT images from a limited number of samples. Indeed, authors in that work also resort to the artificial Shepp-Logan dataset. Thus, I strongly recommend the authors to include this work in their empirical validation. As a note, please correct the reference, as this is an ICCV’21 paper, not arxiv anymore.\n\n- I stress that image reconstruction is not my application domain, but I wonder how useful/reliable 5 and 8 testing images are to draw conclusions about any method.\n\n- Table 2 reports the performance of different methods when 60-views and 120-views are used for measurement regime. I wonder whether this means that the methods are trained with these number of images (i.e,. 60 or 120). If yes, I would argue that the deep learning methods achieve better reconstruction performance on the real dataset. Please clarify whether this is the case (i.e., the number of views is the number of training images across all the methods)\n\n- In both Table 1 and 2, the different versions of the proposed method outperforms classical approaches. Nevertheless, it seems that there is no a clear trend among all these methods (particularly for the real dataset, Table 2). For example, HMC UINR obtains the best NLL performance, while its ECE values largely degrade compared to the ensemble with 10 models (or very similar with MCD UINR). If authors would have to suggest which method to promote for researches wanting to continue in this topic, what would be their suggestion?\n\n- I feel that the supplemental material can be shortened substantially. There are many sections that are well-known or can be found in existing publications/books. Thus, I do not see how they help to the manuscript (beyond avoiding the reader having to search for this content).  ", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "Yes", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "This work presents a first study on the calibration performance of of implicit neural representations for image reconstruction. To this end, authors reformulate the INRs from a bayesian perspective, which allows to quantify the uncertainty on the predictions of the proposed model. For evaluation purposes, two CT datasets are employed, and several existing methods are included in their comparison. \n\nPlease find below my detailed comments for this work.\n", "claims_and_evidence_yn": "Yes", "claims_explanation": "", "audience_yn": "Yes", "audience_explanation": "", "requested_changes": "Please see the list of weaknesses for the potential changes to improve the manuscript.", "broader_impact_concerns": "No concerns.", "additional_comments": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "YusgSered4", "reviewer": "Reviewer_yF4H", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "Strengths:\n- The proposed approach is simple and could benefit medical imaging community working in inverse problem. \n- The presentation and flow of the paper is great.  \n- The experimental design is clear, easy to follow and demonstrate the strength of the proposed UncertaINR. \n\nWeakness:\n- The proposed approach is based on INRs and the extension looks minimal. There could be different ways to extend INRs (choice of priors, choices in architectures, etc.). The paper doesn't seem to acknowledge or discuss these aspects. \n- The paper is missing some critical discussions. ", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "Yes", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "The paper presents a novel method, UncertaINR, based on implicit neural representations (INR), which achieves well-calibrated uncertainty estimates without sacrificing reconstruction quality relative to other classical, INR-based, and CNN-based reconstruction techniques on realistic, noisy, and underdetermined data.", "claims_and_evidence_yn": "Yes", "claims_explanation": "", "audience_yn": "Yes", "audience_explanation": "", "requested_changes": "More discussion is required on the priors. E.g., GOP vs. zero-mean Gaussian prior or any other potential priors. \n\nCan you also clarify on why TV is commonly used in medical imaging. It be helpful for readers not familiar with medical-imagining.\n\nIt is quite interesting to see MC-based approach ourperform deep ensembles. It would be great to have a separate discussion on this aspect. \n\nComputational cost of each of the compared approaches is also missing. Can authors provide them? ", "broader_impact_concerns": "N/A", "additional_comments": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "ToLDBg19jo", "reviewer": "Reviewer_98YR", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "Strengths:\n\n- very well written, I was unfamiliar with CT inverse problems before but the paper gave enough context to follow along and to confirm things with google searches\n- extensive and rigorous ablation study and overall evaluation in main body and appendix\n- touches almost all relevant points one might be interesting in in such a study ....\n- code and data are provided and readable\n\nWeaknesses\n\n- with the exception of an easy runtime and energy comparison (figure 7 shows view dependency, but I would like seeing convergence plots or at least approximate times to convergence of each method, total computational overhead as well as *ideally* energy consumption/carbon impact quantified in the paper. Carbon and energy are optional, but I think adding the other elements to the paper would improve the evaluation)\n- include the standard deviations across results tables please? especially since all activation functions studies appear to have significant std \n- there is no limitations section in main body or appendix or discussion of limitations in the conclusions, which despite the papers thorough investigation appears odd to me (small dataset size, only a single natural dataset, a collection of unanswered questions raised throughout the work would all benefit the consumers of this work in my opinion)", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "Yes", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "The paper performs an extensive empirical evaluation of uncertainty quantification  (UQ) methods for implicit neural representations for CT inverse problems. The paper compares the following UQ methods\n\n-  monte carlo dropout\n- Hamiltonian monte carl\n- deep ensembles\n- bayes by backpropr\n\n\nwhich are evaluated on 1 artificial and 1 real world dataset. The paper perform extensive ablation studies on the artificial dataset and compares against, classical and deep learning baselines, evaluating reconstruction quality as well as calibration of the UQ and finding that their method compares favourably against the baselines, finding in particular that DE and MCD are pareto-strong methods in terms of computational effort,reconstruction quality  and UQ calibration", "claims_and_evidence_yn": "Yes", "claims_explanation": "", "audience_yn": "Yes", "audience_explanation": "", "requested_changes": "Please address the  weaknesses I outlined above, also I believe you meant to write theta and not W in appendix E, right below eq. 62? Otherwise there might be something missing somewhere since I do not see W reappear in that section.\n\n", "broader_impact_concerns": "The use of UQ methods in medical imaging could  increase the trust medical personal places in these methods, which given that while MCD and DE are impressively correlated with error, we still lack (to my knowledge) theory and experience of the interplay between neural method, MCD/DE, data properties and the final UQ. While the work is an important step towards creating this experience, I think the limitations section I raised as missing in weaknesses also warrants a statement about the context in which this work should be placed w.r.t use in real world usage (i.e., make an assessment and official recommendation on whether these results and these methods should be applied in practice and what precautions would be recommended/what future work should be done to avoid over-optimism and potential harm caused by using these methods to speed up CT scans).", "additional_comments": "", "extra_scores": {}, "extra_text": {}}]

Author rebuttal:

Meta-review: All three reviewers assess the paper as "leaning accept" or "accept."

Reviewer yF4H had no concrete points of improvement in the assessment, and Reviewer 98YR had only positive comments.

Reviewer RRpV felt that some of the appendices are unnecessary and could be cut.  I am more neutral about this point.  I am marking the paper as "Accept with minor revision" but I am also satisfied if the authors feel strongly that they would like to keep the appendices included.  This recommendation should be viewed by the authors as indicating that whether or not to make the recommended change is at their discretion, as the article would in any case pass the threshold for acceptance to TMLR in my opinion.
Decision: Accept (poster)