Implicit Neural Representations for Robust Joint Sparse-View CT Reconstruction
Authors: Jiayang Shi
Junyi Zhu
Daniel Pelt
Joost Batenburg
Matthew B. Blaschko
Venue: Accepted by TMLR ()

Abstract: Computed Tomography (CT) is pivotal in industrial quality control and medical diagnostics. Sparse-view CT, offering reduced ionizing radiation, faces challenges due to its under-sampled nature, leading to ill-posed reconstruction problems. Recent advancements in Implicit Neural Representations (INRs) have shown promise in addressing sparse-view CT reconstruction. Recognizing that CT often involves scanning similar subjects, we propose a novel approach to improve reconstruction quality through joint reconstruction of multiple objects using INRs. This approach can potentially utilize the advantages of INRs and the common patterns observed across different objects. While current INR joint reconstruction techniques primarily focus on speeding up the learning process, they are not specifically tailored to enhance the final reconstruction quality. To address this gap, we introduce a novel INR-based Bayesian framework integrating latent variables to capture the common patterns across multiple objects under joint reconstruction. The common patterns then assist in the reconstruction of each object via latent variables, thereby improving the individual reconstruction. Extensive experiments demonstrate that our method achieves higher reconstruction quality with sparse views and remains robust to noise in the measurements as indicated by common numerical metrics. The obtained latent variables can also serve as network initialization for the new object and speed up the learning process.

Published in Transactions on Machine Learning Research (09/2024)
Implicit Neural Representations for Robust Joint Sparse-View CT Reconstruction
Jiayang Shi*
Leiden Institute of Advonced Computer Science
Leiden University
Junyi Zhu*
Samoy..hwu@essa..ku
Center for Processing Speech and Images
KU Lewven
Daniël M. Pelt
d.m.pelt@liccs..eddemmmii
Leiden Institute of Advanced Computer Science Leiden University
K. Joost Batenburg
Leiden Instätute of Advanced Computer Science Leiden Universily
Matthew B. Blaschko
matthew. naatheew.blschhoo@@eeee www.:hheoe...ttt...u...e........ kulewwen..b be
Center for Processing Speech Images
KU Leuven
Reviewed OpenReview: /ttps://openrevieee net/ formm? id= XCzuQIOoXR
Abstract
Computed Tomography (CT) is pivotal in industrial quality control and medical diagnostics. Sparse-view CT, offering reduced ionizing radiation, faces challenges due to its under-sampled nature, leading to ill-posed rnconstruction problems. Recent advancements in Implicit Neural Representations (INRs) have shown promise in addressing sparse-view CT reconstruction. Recognizing that CT often involves scanning similar subjects, we propose novel approach to improve reconstruction quality through joint reconst reconstruction of multiple objects using INRs. This approach can potentially utilize the advantages of INRs and the common patterns observed across different objects. While current INR joint reconstruction techniques primarily focus speeding up the learning process, they not specifically tailored to enhance the final reconstruction quality. To address this gap, we introduce a novel INR-based Bayesian framework integrating latent variables to capture the common patterns across multiple objects under joint reconstruction. The common patterns then assist the reconstruction of each object via latent variables, thereby improving the individual reconstruction. Extensive experiments demonstrate that our method achieves higher reconstruction quality with sparse views and remains robust to noise in the measurements as indicated by common numerical metrics. The obtained latent variables also serve network initialization for the new object and speed up the learning process
"Equal contribution.
WW have used ChatCPT provided by OpenAl assist in writing. The language model was employed at the sentence level tasks such fixing grammar and rewording sentences. We assure that all ideas, claims, and results presented in this work human-saurced.

---

Published in Transactions on Machine Learning Research (09/2024)
1 Introduction
Computed Tomography (CT) is a crucial non-invasive imaging technique, extensively employed in medical diagnostics and industrial quality control. In CT, an object's 's internal structure reconstructed from X-ray projections captured multiple angles, posing a complicated inverse problem. In certain situations, reducing the number of CT measurements can yield advantages such as decreased radiation exposure and enhanced production throughput. However, this sparsity in measurements, along with other factors such the presence of noise and the size of detectors, complicates the reconstruction process, making it an ill-posed inverse problem. Such challenges arise not only in CT reconstruction but also across diverse computational tasks. Therefore, while our study centers on sparso-view CT reconstruction, the core idcas are transferable to mumerous inverse problems, such as Magnetic Resonance Imaging and Ultrasound Imaging.
Different strategies have been developed address the challenges of sparse-view CT reconstruction by incorporating auxiliary information. Supervised learning techniques learn mappings from measurements to images (Zhang et al 2018b; Han & Ye 2018 Zhu al.)
2018 Wu et al. 2021). These methods train on
extensive datascts containing dense measurements, synthesizing the sparse view through physical models of the measurement process. Alternatively, [Song et al. (0222) introduces an approach that directly learns the image distribution from high-quality reconstructed images using generative models. Several diffusion- based
approaches (Chung et al. 2022, Liu et al., 2023 Song et al. 2023 Xu et al. 2024) have been specifically tailored for CT applications, further refining the foundational ideas presented by [Song et al.) (2022) However, their method still relies on large, domain-speeific dataset, that closely matches the target of reconstruction. Dependence on large datasets introduces practical limitations supervised learning techniques and generative methods, particularly when collecting large-scale and high-quality data is intractable, such due to privacy regulation in medical cases or timeliness in industrial settings. It is important to note that supervised learning and diffusion-based approaches represent only a subset of the methods available for CT reconstruction. There
are also some studies that leverage heuristic image priors, e.g. Total Variation (TV) (Sidky Pan 2008 Lit
ctt al.] 2013 Zang al.
2018). While effective, these methods lack the ability incorporate domain-speeifc
enhancement. Another approach involves using few dense-view images as priors for reconstruction (Chen ct al. 2008 Shen et al. 2022 However, this method hinges on the availability of denso-view images similar to the target object, which is often impractical when the appearance of the object to be reconstructed is unknown.
Building on the foundations of CT reconstruction, many works explore the potential of implicit neural representations (INRs). Thanks to the advantages of INRs, such as their inntinuous representation nature
contrast to conventional discrete representations (Lee al. 2021 Xu et al. 2022 Grattarola & Vandergheynst,
2022), these methods have consistently delivered promising reconstruction results with limited number of
mcasurements ZZang et al. 2021 Zha et al.) 2022, Rückert et al. 2022; Wu et al. 2023b). Given INRs' proven capabilities in CT ruction and the known advantages of incorporating prior information, either heuristically through TV (Zang et al. 2021] or explicitly from dense-view CT images (Shen et al. 2022), we soek to morge these two paradigms. Aiming for broad applicability, we avoid reliance on pre-curated datascts. Instead, we draw inspiration from the fact that modern CT machines routinely scan similar subjects, such as patients in hospitals or analogous industrial products. This commonality drives us explore a novel question in this research:
Individual Reconstruction Sinogram INRs
Joint Reconstruction Sinogram
Figure 1: Compared individual reconstruc- tion, joint. reconstruucton enables INRs to share statistical regularities among multiple objects, thereby enhancing the quality of re- construction under conditions of oparse-view CT scans and inherent measurement noise. The examples shown here are CT reconstruc- tions of walnuts from a sparse set of projection angles, with the joint reconstruction results obtained using proposed approach.
Reconst.
INRs
Reconst.
Node
Node 1
Node
Node
2

---

Published in Transactions on Machine Learning Research (09/2024)
Figure 2: Framework of our proposed method. It uses latent variables to capture the rela- tion among all reconstruction nodes. The latent variables are updated based on all nodes and regularize each individual recon- struction via minimizing the KL divergence terms DKL. wy denotes the parameters of j-th node, distributed according to N (HJ, Pÿ), while NNw,, or) specifies the prior distribution of parameters wY.J-
Latent Variable
(Ww
Σj=1 DRI(WWH, pj) py) (l)
W1 ~ N1HL,P)
W2 (22.
w N(HLIP)
O
O Node 2
O Node J
Node 1
Can INRs use the statistical regularities shared among multiple objects with similar represen- Iation to improve reconstruction quality through joint reconstruction?
Figure 1 contrasts the methodologies of individual reconstruction against joint reconstruction. In our explo- ration of this research avenue, we found that several existing methods can be adapted for our purpose (Zhang et al. 2013 Ye et al. 2019 Tancik et al. 2021 Martin-Brualla et al.] 2021 Kundu et al. 2022). These approaches have utilized the inherent statistical regularities across different objects, represented by the network weights in INRs. However, they often focus on addressing issues like convergence rate (Tancik et al.,
2021 Kundu et al.) 2022 rather than optimizing the reconstruction quality itself. As demonstrated in our experimental evaluations $ these methods achieve lower PSNR and SSIM compared to our proposed method in joint reconstruction scenarios.
To address our rescarch question, we propose novel INR-based Bayesian framework, specifically tailored to adaptively integrate prior information inlo network weights during the training process. This is achieved by utilizing latent variables that encapsulate commonalities in the parameter space among multiple objects' neural representations. The latent variables representing prior information are then leveraged to enhance the accuracy of individual reconstructions. The convergence of our model is guided by minimizing the Kullback-Leiblee (KT) divergence between the prior and the estimated posterior distributions of the neural representatin networks. Importantly, our framework can autonatically adjust the regularization strength of the prior information based on the similarity among neural reprosentation networks. Overall, our framework provides robust solution to the challenges posed by sparse and noisy measurements, and varied reconstructions in CT imaging. An illustration of our proposed Bayesian framework is presented in Figure 2
Our Contributions: i) We explore a novel problem of INR-based joint reconstruction in the context of CT imaging, supported by a comprehensive review of existing methods that could be adapted to address this challenge. ii) We propose principled Bayesian framework that adaptively integrates prior information throughout the training process for enhanced reconstruction quality. iii) Through extensive experiments, we evaluate various facets of reconstruction performance using common numerical metrics. Our results establish that our method outperforms the compared INR-based baselines.
2 Related Work
We briefly outline key studies related to our focal areas, with comprehensive understanding of INR and Neural Radiance Fields (NeRF) available in the survey (Tewari et al. 2022 Molaei et al. 2023 The key working principles of INR for CT reconstruction is explained in $3
Coordinate-basedd Multi-Layer Perceptrons (MLPs) intake spatial coordinates and output values, such as RGB or density, processed (hrough MLP. Unlike traditional discrete representations that directly map inputs to outputs pixel grids, these coordinate-basee outputs are generated by the MLP. Therefore, such coordinate-based representations are often regarded as INRs. Originally, INRs faced challenges in capturing high-frequency details. However, the introduction of coordinate encoding, such as Fourier feature transformations (Jacot et al. 2018 Tancik et al. 2020), has significantly enhanced their ability to represent. finer details.
NeRF is a state-of-the-art INR approach that reconstructs scenes from photos taken at multiple angles Mildenhall et al. 2021 Barron et al. 2021 2022), similar to CT where projections acquired at different.
3

---

Published in Transactions on Machine Learning Research (09/2024)
angles to compute reconstruction. While NeRF processes optical images to render RGB values by considering light transmitlance and occlusion, CT reconstruction calculates attenuation coefficients to reveal internal compositions using penetrating X-rays. We therefore briefly introduce the NeRF methods that we adapt as comparison baselincs. Building on the groundorrk, several techniques have emerged to leverage the shared information across different objects or scenarios. NeRF-wild (Martin-Brualla al. 2021 differentiated between static and transient scene aspects, an approach echoed video representations (Li al. 2021 Mai & Liu 2022). [Tancik et al. (2021) introduced meta-learning techniques to learn a meta initialization which converges faster than training from scratch. They have also applied their method to CT reconstruction. For the same purpose of accelerating convergence, Kundu et al. (2022) introduced federated learning to obtain
the meta initialization. Chen & Wang [2022] proposed using transformers as the meta-learners.
INRs' potential in CT reconstruction has been exploited in various ways. Sun et al. (2021) focused on representing sparse measurements, Zang et al.
22021) combined INRs with TV and non-local priors for CT
reconstruction. Notable advancements include cone-beam CT optimization (Zha ct al.)
2022) and adaptive
hierarchical octree representation (Rückert al. 2022 Wu al.
(023b) proposed to improve reconst ruction
precision by reprojecting inferred density fields. Shen et al. 2022 proposed to pre-train the INR network with
high-quality prior image. Reed et al. (2021) incorporated a scene template and motion field in roconstructing
time-varye varying CT. Furthermore, Wu et al. (2023a explored unsupervised INRs for metal artifact reduction in CT imaging.
3 Problem Statement and Preliminaries
The CT acquisition process, in an ideal noiseless scenario, can be mathematically described by the equation: y - Az, where æ E IRm represents the unknown object of interest and y E R" symbolizes the ideal noiseless measurements. In practice, the actual obtained measurements ÿ E R" are often noisy, where the difference between actual and ideal measurements noise E R" is stochastic term representing general noise, including non-additive types like Poisson noise, which is the primary focus of this study. For simplicity and to maintain focus the met hodologica aspects, this manuscript primarily expands formulas under the assumption of noiseless measurements unless specified otherwise.
ý y is the measurement noise. We note that the measurement
These measurements arise from the interaction between the measurement matrix A E Raxm and the object. The task in CT is infer the unknown object from the acquired CT measurements y. The inherent challenge lies in the common sparsity of these mcasurements, resulting in problem ill-posed. Meanwhile, measurement noise can further corrupt the reconstruction results.
This makes the reconstruction
The INR designed for CT reconstruction is a function fw : 3 + R parameterized by w. It maps the threc-dimensional spatial coordinates of the object to its intensity. INR consists of two components, formulated as fw = MM y. Here, y R3 + Rd servers as the position encoding (Tancik et al. 2020 Barron et al. 2022 Martel al. 2021 Müller et al. 2022), while M Rdd Rd acts as the neural representation. Typically, M is an MLP.
The function (-) takes coordinate c, E 3 and maps it to the intensity value v E R.. For full set of coordinate C = {c1, c ., ]. ) the INR outputs the representation of the entire object as (C) = [fw
[v(())). The network's parameters w can be optimized by minimizing the loss function: f(w) = | (C)-
Joint Reconstruction Problem. We aim to simultancously recover objects Z1:7 using their corresponding measurements y1. and measurement matrices A The joint reconstruction problem can be formulated as:
wi.j arg min
L (w;); j=1
(1)
WiN
& (,(,)) = || w (C) y
(2)
While this formulation allows for different measurement matrices for each object, we primarily consider scenarios where all objects share the same measurement matrix A in our experiments. A straight way
4

---

Published in Transactions on Machine Learning Research (09/2024)
to solve Equation [ is by optimizing the J nodes separately, akin to individual reconstruction. However, we propose introducing a dynamic prior, which links all the models wi.J during training and updates in response to their optimization. This Bayesian framework is designed to exploit shared statistical regularities among different objects, thereby enhancing the quality of the joint reconstruction.
3.1 Existing Methods Available for Joint Reconstruction
In our exploration of this research avenue, we found several existing methods, though originally designed for different problems, have utilized the statistical regularities of multiple objects. These methods may be adaptable to our research problem. In this section, we delve into these methods in greater detail. Empirical evaluations, as demonstrated in $5 suggest that some of these techniques can outperform the individual reconstruction approach. However, they do not fully exploit the statistical regularities to improve the reconstruction quality, which we discuss in the $
Composite of Static and Transient Representations. Martin-Brualla et al. (2021 introduce a composite representation approach NeRFWild, designed manage variable illuminations and transient occluders collective observations. While CT does not involve variable illumination, their concept of combining "static" and "transient" components can be adapted for our context, which we term INRWild.
Let GΦ represent the neural representation for the static component (Ø is the weights of MLP g) and H.r signify the transient component (w is the weights of MLP H). For a given set of J objects, each object-associated reconstruction node has its distinct transient network wy and corresponding transient feature b,. In contrast, the static network & shared across all nodes. The objective for this framework is formulated as:
E1A IA, (Hw, (C)) ((0C))+-(000)) ((()) y3112.
arg min w. bid
The output of the static network G6 is split into two components:
(C) which represents the static intensity, and G) (C), which serves as intermediate features for the transient networks. The symbols and \r denote the division of outputs into these two components. Detailed explanation and framework schematic are in Appendix B.2 At its core, INRWild emphasizes training the static network Φ, which embodies most learnable parameters, using aggregated losses. Concurrently, the individual representations, characterized by w1.J and bı., are refined based on & that corresponds to the static part. W1:J, b and & are jointly optmized achieve the composition of static and transient parts.
Model-agnostic Meta-learning (MAML): Meta-learning aims to train network in way that it can Fallah et al. 2020). Several INR-based works have employed MAML to obtain meta- learned initialization thereby accelerating the convergence or enabling model compression (Tancik et al. 2021 Lee et al. 2021 In the MAML framework, computational cycles are organized into "inner loops" and "outer loops", indexed by A ==11.. 1....K and t = I,..., T respectively. For each node j = J, the networks Wi.J are initialized according to the meta neural representation 1ho = 0. These networks then undergo K steps of inner-loop learning:
quickly adapt to new tasks (Finn et al. 2017 Nichol et al. 2018
w//h) = w (k-1) V wy l (10) (1)),
where 77 is the inner-loop learning rate. After these K steps, the meta network updates as follows:
(K), Ow)(K)
@ = 0(-11) 1
(4)
00
where a is the outer-loop learning rate. Equation is form that can be further specialized. After T steps of outer-loop optimization, the meta-learned neural representation (()) serves as an effective initialization for individual reconstructions.
Federated Averaging (FodAvg): Kundu et al. (2022) suggested employ FedAvg (McMahan et al. 2017 as the optimization framework of the meta-learned initialization. Like MAML, FedAvg also consists inner and outer loops. The inner loop is identical to MAML. Whereas, the outer loop simplifies the optimization by
5

---

Published in Transactions on Machine Learning Research (09/2024)
(K) averaging all individual nelworks, represented as 0 3 E, w) w Essentially, the meta network acts as the
centroid of all networks. +
4 A Novel Bayesian Framework for Joint Reconstruction
In this section, we introduce INR-Bayes, our Bayesian framework for INR joint reconstruction.
Motivation. NeRFWild is proposed to address scene reconstruction challenges, such as reconstructing popular sightseeing sites from in-the-wild photos. These scenarios often involve a primary larget object, like a landmark, amidst. various transient elements such as pedestrians and changing light conditions. A method that uses a composition of static and transient components operates under the assumption that the core representations of the those of the main object- substantially overlap, despite different viewpoints or transient changes. This assumption typically holds true in 3D scene reconstruction, where observations are collected from multiple perspectives of the same object. Our empirical findings INRWild indicate such methods do not work efficiently in CT reconstruction and other image-level reconstruction tasks (discussed in 5 Meta-learned initialization methods, such as FedAvg and MAML, train a meta model to capture a high-level representation, which can then be promptly adapted to individual objects. Although the meta model can effectively extract statistical regularities among multiple objects, we observe that this prior information tends to be lost during the adaptation phase. This loss occurs as the adaptation relies solely on local measurements, leading to an overfitting issue, as we will discuss in 55 Overfitting is a notorious problem in iterative methods of CT reconstruction (Herman & Odhner 1991 Effland et al.) 2020), which renders those methods sensitive to hyperparameters, such early stopping. This sensitivity consequently increases the difficulty of deployment.
To enhance the joint reconstruction process, we propose a principled Bayesian framework. Our method employs latent variables, denoted by (w. ; to capture commonalities among individual INR networks. These latent variables actively serve as references throughout the entire reconstruction process, guiding the training of individual networks.
Definition and Notation. We introduce distributions to the networks wi:J for J objects, and define latent variables (w,r) that parameterize an uzis-aligned mullivariale Gaussian prior N(w, .r) from which the weights WI.J are generated. Each node is assumed to be of the same size. A Gaussian distribution is a natural and effective choice for the weight distribution of the MLP such as INR networks (de G. Matthews let al.] 2018). The latent variables collectively serve to capture the shared trends within the networks, effectively quantifying the mutual information across different objects. To simplify the model, we assume the conditional independence among all objects: p(w1:J| [w, o)- = - p(wy/,, σr. This assumption of conditional independence allows
decompose the variational inference into separable optimization problems, thereby facilitating more efficient parallel computing.
Given that the measurements of the objects yı,..., yy are mutually independent and that cach network focuses on a specifie object, the posterior distribution of network weights and latent variables can be derived using the Bayes' rule as p(WL.J, (.l.J.,.olyy... olyi.)) x p(w. o) II; p(y, (w))p(pwlle σ). While this posterior enables various forms of doductive reasoning such as reconstruction uncertainty quantification, inferring the truc posterior is often computationally challenging. Moreover, the selection of an appropriate prior p(w, σ)) poses its own difficulties (Wenzel et al. 2020; Fortuin et al. 2022 To tackle these issues, we present an algorithm that aims at maximizing the marginal likelihood p(y w, σ), integrating out WI.J. Detailed derivations are provided in Appendix A
4.1 Optimization Algorithm
We approximate the posterior distribution of the network weights Wi.J using variational inference techniques Kingma& & Welling 2013 Blei al.) 2017). Specifically, we introduce the factorized variational approximation g(w)))) = II ((w)), employing an axis-aligned mallivariate Gaussian distribution for the variational family, i.e q(w,) = ((, P.).
We note that AvdAvg can also be regarded as using a specific first-order algorithm of meta learning called Reptile Nichho et. al. 2018: and setting the loop learning rate 1.
6

---

The minimization of the negative log- ligllill term is achieved through the minimization of the squared error loss of reconstruction (see Equation (2). The KL. divergence term Dxz serves as regularization constraint on the network weights, pushing the posterior ((w,) to be closely aligned with the conditional Published in Transactions on Machine Learning Research (09/2024)
Variational Expectation Maximization. To optimize the marginal likelihood p(y1. |w, σ)) we maximize the evidence lower bound (ELBO))
ELBO (g(w):)), w, σ) = )g(w)))) log P(y.. wid (w))
(5)
((W1:))
The ELBO is optimized using Expectation Maximization (EM) (Dempster et al. 1977), a two-stage iteralive algorithm involving an E-step and an M-step, Generally, each EM cycle improves the marginal likelihood p(y,J/, o) unless reaches local maximum.
E-step. At this stage, the latent variables {w, a) are held fixed. The aim is to maximize ELBO by optimizing the variational approximations g(Wr.)). By the conditional independence assumption, the objective can be separately optimized. Specifically, each network j minimizes:
L (g(wr,) = -Bq(w,) log p(y,/w,)) + (xz(a(w,)) (p((w, w w.))).
prior determined by {w,a), which represent the collective mean and variance all the networks in the ensemble. The KL divergence term thus serves to couple the newral representations across networks, allowing them to inform each other.
M-step. After obtaining the optimized variational approximations q(w):)) we proceed to maximize the ELBO with respect to the latent variables {w,
ELBO(W,Or or) x
Σ Egg log logp(u;)k,,o)
Equation
7 allows for a closed-form solution of {wo oJ derived by setting the derivative of the ELBO to
zero:
Σ pj: o' 1 = Σ + (Fg -w
w
(8)
In our framework, w serves as the collective mean of individual network weights, while or provides a measure of dispersion. This measure factors in both individual variances and deviations from the collective mean. We note that the KL divergence term, introduced in the preceding E-step objective (see Equation (6), operates wise. As a result, during the training process, weight elements with larger a values are less regularized, and vice versa. This results in flerible, self-adjusting regularization scheme that adaptively pushes all weights loward the latent mean, w.
4.2 Algorithm Implementation
Next, we discuss the intricacies of implementation, addressing in particular the computational challenges associated with Equation ( See Algorithm [ for summary of method.
Variational Approximation and Reparameterization Tricks. Given that the expected likelihood in Equation 6 is generally intractable, we resort to Monte Carlo (MC) sampling to provide an effective estimation. Moreover, we introduce an additional hyperparameter / for the KL divergence to balance the tradc-off between model complexity and overfitting. Linking the likelihood with the square error loss, for any node j, the effective loss function can be expressed as:
L (q(w,)) ~ [F 10, (C)) , + 3D (0Dx(a((;)))p(p) o)),
where S denotes a sample from (w;)). We only do MC sampling once at each iteration, which works efficiently in practice. To facilitate the gradient-based optimization schemes, utilize the reparamederization

---

Published in Transactions on Machine Learning Research (09/2024)
Algorithm INR-Bayes: Joint reconstruction of INR using Bayesian framework
Input: Mi:J" π I:J w σ", 17, B, T, R
(0.0)
Output: Mi:J TI:J
(R.T) (R.T) w R
1: for r= to R do
for j......,, in parallel do NodeUpdaaeew"" "0"=1
4: After each E-slep, collect Pi:s T:J
(n.T) (n.T)
5:
b Compute the optimal latent variable w, a.
w" - = (r:2)
o² = --= log (1 exp(t) + (11) (1)):))-c =
7:
NodeUpdatoow".. .o"): for -= 1...,, T in parallel do
10:
Sample mj.
e ~M; (1) + log (1+ exp((T)))))) N 10,1
11:
12:
> Compute the loss function.
4
13:
((H):P)) = ||A (C)) (C))
1113+
14:
SGD on pb Pj with learning rate 17. (r,t1) (r.t)
pij n 0C (r.t1)
; (r.0) -17
15:
Mi
us T
trick (Kingma & Welling 2013 (ww)) = M4 + log (1 + exp (T,)) I). Here, additionally employ the softplus function in parameterizing the variance Pj with the variable и to ensure the non-negativity of the variance.
The EM algorithm operates by alternating between E and M steps. In the E-step, we perform T iterations to achieve locally optimal variational approximations. Following this, the M-step utilizes the closed-form solution (see Equation ( for efficient parameter updating. The entire cycle is exccuted for rounds to ensure convergence. Finally, the posterior means MM.J serve as the weights for individual neural representations, while the prior mean is used the weights for the latent neural representation.
5 Experiments
Dataset. Our study utilizes four CT datasets: WalnutCT with walnut scans (Der Sarkissian et al. AluminumCT of aluminum alloy at different fatigue-corrosion phases from Tomobank (De Carlo et al. 2018
2019 )
LungCT from the Medical Segmentation Decathlon (Antonelli et al. (002) and 4DCT on the lung area (Castillo al. 2009p. Additionally, we include natural image dataset CelebA (Liu et al. 2015 to evaluate the generalizability broader applications. The objects of interest centrally positioned within the images across all datasets. Central positioning in practice is achievable during acquisition by monitoring initial placement.
Comparison Methods. We compare our approach with other methods that do not require a set of densely mensured data, thereby excluding those of supervised learning: i) Classical techniques: Filtered Back Projection (FBP) and Simultaneous Iterative Reconstruction Technique (SIRT) (Gilbert) 1972); ii) Regularization- based approaches with model-based reconstruction method FISTA (Beck & Teboulle 2009): total generalized variation denoted RegTGV proposed by Niu et al. (0014) wavelet based regularization RegWavelet (Zhang ct al. 2018a Garduño et al. 2011) ) and second order Lysakcn-LLunderoldd-Tn with Seeti--Oshr-Fatttee TV, denoted as RegLLT-TV (Kazanlsev el al. 2017); iii) Naive INR-based single reconstruction method, denoted as SingleINR; iv) FedAvg, a federated averaging approach proposed by Kundu et al. (2022); v) MAML, a meta-learning technique as discussed by Tancik et al. (2021): vi) INRWild, method adapted from NcRFWild -Martin-rrualla al. 2021). FBP and SIRT are classical methods that do not use networks, while all other methods employ identical INR network as described in the next paragraph.
8

---

Published in Transactions on Machine Learning Research (09/2024)
INR Network Configuration. The same backbone and associated configurations are applied to ensure fair comparison. All INR-based methods employ the STREN architecture (Sizzmann et al. 2020) coupled with the same positional embedding (Tancik et al. 2020). Specifically, we consistently use a Siren network with the following dimensions: a dopth of 8 and width of 128 for the WalmutCT and AluminiumCT experiments, and a depth of 8 with width 256 for the rest of the experiments. Position encoding is implemented using Fourier feature embedding (Tancik et al.) 2020). with an embedding dimension of 256 for WalnutCT and AluminiumCT, 512 for the rest. Consistently, this embedded position is input to the INRs to facilitate density prediction. In alignment with Martin-Bruunt Brualla et al. (2021) INRWild utilizes 8-layer SIREN network for the static segment and 4-layer SIREN for each transient component. We note that all nodes use the same size network for our method and comparison methods.
CT Configuration. We simulate CT projections using the Tomosipo package (Hendrissnn et al. 2021 with a parallel beam. Experiments on 4DCT, CalnutCT and CelebA use projections from angles across 180", while LungCT uses 60 angles and AluminiumCT uses 100 angles. Practical applicability is further tested under 3D cone-beam CT setting, detailed in Appendix D.1
Experiment Configurations. Intra-object: 10 equidistant slices from an object's center. ii) Inter-object: 10 slices from different objects, cach from a similar vertical position. iři) 4DCT: 10 temporal phases from one 4DCT slice. All INR-based hods undergo 30K iterations, with MAML using the first 10K and FedAvg the first 20K for meta initialization, then proceeding to adaptation. For classical methods, FBP_ 2P_C computes the FBP reconstruction while SIRT_ SIRT_CUDA CUDA from the Astra-Toor Toolbox used for SIRT, set to run for 5,000 iterations. Hyperparameters like learning rates are optimized using a subset of data (details in Appendix B.4].
Metrics. We primarily evaluate using Peak Signal to Noise Ratio (PSNR) and Structural Similarity Index Measure (SSIM), with metrics referenced against ground-truth images. We calculate mean and standard error over all reconstructed images in each experiment.
5.1 Results
Method
Metries
Inter-walnut
4DCT
Intra-lung
Inter-Aluminium
Inter-Lung
Inter-Faces"
FBP
PSNR
19.54 +0.25
25.19 #0.03
26.50
#0.06
32.13 =0.05
24.97 #0.14
18.12 #0.39
SSIM
0.328
#0.003
0.529 #0.001
0.568 40.002
0.717 #0.002
0.503 10.004
0.549 #0.006
PSNR
23.82 40.25
28.61 10.03
28.81 #0.06
28.61 10.33
28.32 10.13
29.13 #0.20
SIRT
SSIM
0.435
20.007
0.746 #0.001
0.719 #0.002
0.836 10.001
0.678 20.005
0.766 20.005
RegTGV
PSNR
30.64
40.33
31.50 #0.04
31.46 #0.07
27.21 #0.03
31.18 20.19
28.39 #0.18
SSIM
0.872 #0.005
0.859
#0.001
0.810
40.002
0.868 #0.001
0.809 #0.006
0.833 #0.005
PSNR
24.79 10.25
27.00 +0.03
27.95 20.05
25.75 20.03
27.76 20.11
27.00 20.21
Reg Revelvelet
SSIM
0.460
#0.006
0.594 #0.001
0.634
40.002
0.817 #0.001
0.626 #0.001
0.701 #0.006
RegLLT-TV
PSNR
33.33
40.30
32.39
#0.04
32.26 #0.08
27.19 #0.03
31.95 20.19
30.09 #0.22
SSIM
0.933
#0.002
0.890 #0.001
0.828 40.002
0.867 +0.001
0.828 +0.006
0.858 #0.005
SingleINR
PSNR 35.18
#0.43
34.07
#0.04
32.80 #0.11
35. 35.62 =0.06
32.64 =0.27
30.90 =0.32
SSIM 0.934
#0.004
0.877 #0.001
0.815 40.002
0.850 10.001
0.821 10.007
0.831 10.008
INRWild
PSNR 19.76 40.35
33.76 +0.04
28.46 20.07
29.77 10.05
25.05 10.18
16.43
=:0 28
SSIM
0.675 20.010
0.863 10.001
0.674 40.003
0.694 20.002
0.560 20.08
0.260 20.011
PSNR 35.66 +0.38
34.35 +0.05
33.26 10.10
36,22 #0.06
33.13 #0.22
25.71 =
MAML
SSIM
0.941 +0.003
0.881
40.001
0.825
40.002
0.871 40.001
0.833 10.006
0.647 10.012
FedAvg
PSNR
31.57 40.32
34.67
+0.04
32.42 20.08
35.75 20.06
31.68 #0.19
30.7 74 #0.29
SSIM
0,886 #0.004
0.885
+0.001
0.808
+0.002
0.849 +0.001
0.807
10.006
0.827 #0.007
INR-Bayes
PSNR 36.13
10.33
34.79
+0.04
33.90
10.10
36.67
20.08
33.75
= +0.20
31.31 #0.31
SSIM 0.946 10.003
0.894 4000011
0.840 400000
0.881 4.0.001
0.847
+.0006
0.847 10.007
Table 1: Performance comparison across different settings on noiseless measurements. The highest average PSNR/SSIM values are highlighted in bold, consistently showing that INR-Bayes outperforms other methods. "Reconstruction of human faces is included to illustrate our method's broad applicability, despite lacking
practical relevance in physical contexts.
9

---

Published in Transactions on Machine Learning Research (09/2024)
SIRT
RegLLT-TV
SingleINNR
INRWid
MAML
FedAvg
INR-Bayes
12.12
20.45
36.74
22:56
36,35
37.08
S7/69
Wainut
0.570
0.341
0.724
0,562
0.96B
36
A alloy
0.87
D.669
0.257
0.673
27.90
33.06
35.51
27.76
34.15
$9.22
36.63
Lung
0,722
0.308
0.00
D.18
0.585
0.824
0U'4
3197
20.0
10
2645
5556
Face
Cleb
Figure 3: Visual comparison reconstruction performance on noiseless measurements. Enlarged areas are highlighted in red insots. PSNR values are on the top left, with SSIM values on the bottom left. Reconstruction of human faces is included illustrate our method's broad applicability, despite lacking practical relevance in physical contexts. Illustrations figures (first column) are modified from J.Dnesn (2009 Agashi5859 2014
Lene 2021
Liu et al. 2015
Reconstruction Performance. Table 1 shows that our method consistently achieves the highest average metrics across datasets on noiseless measurements, with the exception the SSIM values on the CelebA dataset, excelling in leveraging inherent trends in slices. This advantage particularly pronounced in experiments with distinct transition patterns, as seen both inter-object and intra-object settings. In contrast, INRWild generally underperforms compared to SingleINR, Its strategy of jointly reconstructing static representations while independently capturing transient characteristics fails to enhance overall reconstruction quality. In most scenarios, FedAvg is outperformed by SingleINR, suggesting its inability to learn a meta representation that meaningfully improves reconstruction quality. MAML ranks as the second- method in most scttings, indicating its learned meta reprosentation aids in individual reconstruction performance. However, regularization--ben
methods typically underperform compared to INR-based methods, except on the CelebA dataset where RegLLT-TV achieves the highest SSIM value. This underscores the efficacy of the heuristic LLT-TV prior for natural images as opposed CT images.
The visual comparisons in Figure 3 further validate our results. Reconstructions by SingleINR exhibit noticeable artifacts, while FedAvg and MAML struggle to capture finer image details. RegLLT-TV often results in overly smooth reconstructions, but in contrast, INR-Bayes achieves superior visual quality, effectively balancing smoothness and detail preservation, yielding results closer to the ground truth.
Overfitting Iterative reconstruction methods tend to overfit when applied limited data (Herman Odhner 1991 Eflland et al., (2000) (particularly noticeable in sparse-view CT scenarios). In contrast, Bayesian
frameworks have demonstrated robustness against overfitting (MacKay 1992 Neal 2012 Blundell et al. 2015). To validate this, we extend the training iterations in Table from 30K to 60K. As shown in Figure on inter-object LungCT, the learning curves of basclines deteriorate in the long run, indicating overfitting
"In the context of C'I' reconstruction, conventional iterative methods like SIRT typically approach good approximation to the exact solution early in iterations and subsequently diverge from it, often attributed to convergence issues (Eliving et al. 2014). In contrast, the machine learning domain, overfitting refers scenario where neural network excessively fits training data, leading drop in actuall performance despite decreasing training loss (Srivastava al. 2014). Our experiments contain both non-learning-based methods (SIRT) and learning- based methods (INR). We acknowledge the subtle differences
between convergence challenges and overfitting but the overfitting broadly describe performance deterioration in both noiseless and noisy scenarios.
10

---

Published in Transactions on Machine Learning Research (09/2024)
36
28
PSNR ddB
32
further adaptation
26
PSNR
INR BaRBaays
24
INR Bayes
28
MAML
MAMI
FedAvg
22
FedAvg
26
SingleINR
SingleINR
10k
20k
30k
40k
50k
60k
10k
20k
30k
Total Iterations
Total Iterations
Figure 4: After the initial 30,000 iterations, each joint reconstruction method undergoes an additional 30,000 iterations for further adaplation.
Figure 5: The training curve of different methods on noisy measurement. SingleINR, MAML, and FedAvg overfit strongly in 30,000 iterations.
FBP
SIRT
RegLLT-TV
Single INR
FedAvg
MAML
INR-Bayes
GT
Wainut
27 45
27.0 56
22.22
80.0 D
28 75
ac DC
25
N allay
0.52
0.748
0.633
0.642
0.633
Lung
0.8
23,
20
30.80
91.77
2641
30.21
32
le
OLDO
0.729
DISS
O.6G
0.7G3
Figure 6: Performance comparison on inter-object noisy measurements, Enlarged areas are shown in red
insets. PSNR and SSIM values are provided in the top left and bottom left corners, respectively. Illustrations
figures (first column) are modified from J.Dnesm (2009): Agashi5850) (2014); Lene (2021): Liu al. (2015).
during optimization. Conversely, our approach maintains consistent level of reconstruction quality once the optimal performance is achieved, underscoring its robustness.
Given that practical measurements often contain noise, leading to poorer reconstruction quality, we evaluate the methods on noisy measurements on AluminumCT, WalnutCT, LungCT and CelebA datasets in inter- object setup. The noise is simulated following Hendriksen et al. (2020) (detailed in Appendix B.I.. As indicated Table our method's performance notably superior against other methods under noise conditions. Figure 5 shows the training curves on noisy WalnutCT data. Although all methods reach similar peak PSNR values, the compared baselines rapidly deteriorate due to overfitting, while our method exhibits minimal overfitting thanks to its contimuous prior guidance. The reconstruction outcomes for these noisy scenarios are depicted in Figure 0 This robustness is especially critical in practical scenarios, where determining an exact stopping criterion is challenging, as the ground truth reference is not available.
Challenge in Determinating Appropriate Stopping Criterion. We find that in the noisy CT reconstruction environments it inherently challenging to determine the optimal stopping point when the overfitting problem presents. This due to the variation of convergence speed across different objects. To demonstrate that, present the individual SingleINR training curves from three different patients within
11

---

Published in Transactions on Machine Learning Research (09/2024)
Noisy Walnut
Noisy Lung
Noisy Alumnium
Noisy Faces
Method
PSNR
SSIM
PSNR
SSIM
PSNR
SSIM
PSNR
SSIM
FBP
13.59 20.26
0.270
20.002
20.33
#0.17
0.353
20.003
27.77
=0.04
0.515
#0.001
17.81
=0.38
0.474
20.007
SIRT
23.48 40.25
0.405
20.007
28.03
#0.14
0.623
=0.005
28.23
=0.03
0.759
#0.001
26.90
#0.21
0.681
#0.007
RegTGV
27.76 +0.31 0.748
10.010
27.44
10.17
0.619
=0.006
27.14
=0.03
0.862
#0.001
27.97
#0.18
0.800
#0.004
Reg Wavelet
23.49
+0:25
0.393
20.006
26.52
#0.15
0.548
=0.005
25.19
#0.03
0.716
20.001
22.83
#0.26
0.471
#0.010
Reg..IT-TV
28.51
40.31
0.817
+0.010
30.58
10.17
0.783
=0.006
27.23
#0.03
0.863
#0.001
28.00
=0.18
0.801
40.001
SingleINR
22.03 +:039
0,809
#0.006
27.16
#:023
0.681
=0.008
29.50
#0.12
0.602
#0.006
27.41
#0.24
0.701
#0.009
MAML
26.55 +0.30
0.860
#0.004
30.04
+0.20
0.758
=0.006
32.58
=0.07
0.741
#0.003
27.55
#0.26
0.701
#0.01
FedAAg
25.37 40.30
0.836 +0.005
27.13
+0.22
0.696 =0.006
29.87
10.10
0.617
#0.003
25.67
#0.15
0.669 40.007
BRR-Bayes..
30.13 #0.31 0.885 +0.008 31.20 =0.20
0.799 =0 006 35.09
#0.12
0.869 +0 006 28.24
#0.27
0.729 #0.010
Table 2: Comparison of reconstruction performance noisy measurements on AluminumCT, WalnutCT, LungCT and CelebA dataset.
the noisy LungCT dataset in Figure 7 Despite originating from the same datasel, these three cases exhibit
distinctly different optimal stopping points, suggesting that a mee-izze-itss-ll approach to training termination is ineffective for appraches suffering from the overfitting problem.
30
Figure 7: SingleINR training curves for three different patients from the noisy LungCT datasct, each show- ing unique optimal stopping points. This variability exemplifies the challenges in universally determining the precise moment to stop training.
PSNR/ dB
25
Case
Case
Case
10k
20k
30k
Total Iterations
A Applying to Unseen Data using Learned Prior. To evaluate the generalizability of the learned priors, we apply meta models or latent variables from the inter-object LungCT experiment to guide the reconstruction of new subjects in the dataset. Specifically, we select 5 consecutive slices from new objects, choosing slices from a similar location (i.e. the similar axial slice index) the prior has been trained. The prior information is solely utilized to guide the reconstruction and is not updated during the process. Table 3 shows that FedAvg fails to improve the reconstruction quality compared with singleINR, suggesting its learned meta neural representation struggles to generalize to unseen data. In contrast, both MMR-Bayes and MAML effectively leverage their trained priors for improved reconstruction, with our method showing notably better metrics. Applying learned prior can also accelcrate the training process. Figure X presents learning curves of different methods. All joint reconstruction methods converge faster than individual reconstruction, Initially, FedAvg converges the fastest, but training progresses, both MAML and -NR-Bayes surpass it. Additionally, the results reconfirm the robustness of INR-Bayes against overfitting, an issue that tends to challenge other compared methods.
Table 3: Adaptation on new patients using priors learned from other patients compared to individ- ual reconstruction. The results are averaged by 10 new patients (5 slices each patient...
SingleINR
FedAvg
MAML
INR-Bayes
PSNR 30.22 20.12 29.89 #0.16
30.42 #0.14
31.31 10.12
SSIM 0.769 10.003 0.754 40.004
0.770 20.003
0.799 10.000
Comparison with Different Numbers of Angles. Nexl, we investigate how different methods fare with varying levels of data sparsity. Figure demonstrates that all methods benefit from increased scanning angles. Methods that leverage prior information like FedAvg, MAML, and INR-Bayes outperform singleINR when the number of angles is limited. With only 20 angles, FedAvg's performance on par with our method, indicating that the averaging scheme can be effective in extremely data-scarce scenarios. However, the number
12

---

Published in Transactions on Machine Learning Research (09/2024)
40
30.0
35
27.5
PSNR /dB
INR-Bayes
25.0
PSNR
30
MAML
FedAvg
22.5
INR Bays
25
INRwild
MAML
SingleINR
20.0
FedAvg
SIRT
SingleINR
20
FBP
17.5
10k
15k Total Iterations
25k
30k
20
30
40
50
60
70
80
90
100
Number of Angles
Figure 8: Reconstruction of new patients using the learned prior from 10 other paticnts, compared to individual reconstruction. All joint methods recon- struct. faster with the priors they learned.
Figure 9: Performance across different numbers of scanning angles. Our method shows an advantage over other comparison methods, especially in sparse angle cases.
31.5
The performance of different methods on new patients using prior obtained with different numbers of paticnts. SingleINR presented as a reference. The performance of our propsoed method gradually increases with the increasing number joint reconstruction nodes.
Figure 10:
31.0
PSNR dBB
30.5
30.0
Bayes
29.5
MAML
FedAvg
29.0
SingleINR
5
10
15
20
25
30
35
40
Number of Nodes
of angles grows, both INR WRRBaayes and MAML surpass FedAvg. Tt is also worth noting that the performance gap between singleINP INR and INR-Bayes narrows more data becomes available, suggesting that while the prior information is useful in sparse data siluations, its advantage diminishes in the data-rich environment. Nevertheless, our INR-Bayes method generally yields the best results in various settings.
Impact of Joint Nodes Numbers on Learned Prior. We explore how varying the number of nodes influences the learned prior in an inter-object LungCT dataset setup. Figure 10 illustrates that our INR-Bayes method benefits from increased number of nodes, unlike FedAvg, which consistently lags behind in performance. Notably, MAML shows decline in performance with more nodes, consistent with the trends observed in Figure 17 These findings suggest that INR -NR-Bayes effectively develops a stronger prior with more nodes. As the number of nodes increases, the prior's mean and variance gradually align with population statistics, thereby enhancing its utility for new object reconstructions. This advantage expected to continue until reaching a saturation point with a sufficiently large number of nodes.
Computation. Our method takes 22.7 minutes average for 501 501 reconstruction A100 GPU, 15% longer than other INR methods. This computational overhead yields noticeable gains in reconstruction quality and robustness across diverse settings. We also investigate the computational efficiency and performance trade-off in Appendix D.7
6 Conclusions and Limitations
In this work, we study the possibility of improving the CT reconstruction quality through joint reconstruction and introduce a novel INR-based Bayesian framework. Through extensive experiments, our method has effectively showcased its ability leverage the statistical regularities inherent in the sparse and noisy measurements of multiple objects improve individual reconstructions. This capability allows our approach to outperform compared methods in reconstruction quality, robustness to overfitting well as generalizability:
13

---

Published in Transactions on Machine Learning Research (09/2024)
While the primary focus of our method has been joint CT reconstruction, its underlying principles hold potential applicability across variety of inverse problems that are challenged by sparse measurements.
We recognize that INR-based methods outperform conventional ones but require more computation, making their efficiency a crucial focus for future research. Additionally, the metrics employed in our study may not always correlate with clinical evaluations (Renieblas al., 2017 Verdun et al., 2015). Tf applied in medical fields, clinical verification of our method remains essential understand its practical implications and efficacy.
Acknowledgments
Jiayang Shi, Daniël M. Pelt, K. Joost Batenburg and Matthew B. Blaschko are supported by the European Union H2020-MSCA-ITN-2200
under grant agrocment no. 956172 (xCTing). Matthew B. Blaschko and Junyi Zhu received funding from the Flemish Government (AI Research Program) and the Research Foundation Flanders (FWO) through project number GOG2921N.
References
Agashi5859. After flatting aluminum rod. htttes///ooomnns..kiiiiaaaaaaaaa ffattting..aaat_n___ee aluminum rod.jjp.. 2014. URL htttps:/crreaiiveeoomort org//Lieesss/byysa//... Accessed: date- of-access. Licensed under CC BY-SA 3.0. Modifications were made.
Anastasios N Angelopoulos, Amit Pal Kohli, Stephen Bates, Michael Jordan, Jitendra Malik, Thayer Alshaabi, Srigokul Upadhyayula, and Yaniv Romano. Image-to-image regression with distribution-free uncertainty quantification and applications in imaging. In International Conference on Machine Learning, pp. 717-730.
PMLR, 2022.
Michela Antonelli, Annika. Reinke, Spyridon Bakas, Keyvan Farahani, Annette Kopp-Schneider, Bennett A Landman, Geert Litjens, Bjoern Menze, Olaf Ronneberger, Ronald M Summers, et al. The modical segmentation decathlon. Nature communications, 13(1):4128, 2022.
Jonathan T Barron, Ben Mildenhall, Matthew Tancik, Peter Hedman, Ricardo Martin-Brualle and Pratul P Srinivasan. Mip-nerf: A multiscale representation for anti-aliasing neural radiance fields. In Proceedings of the IEEE/CVE International Conference on Computer Vision, pp. 5855-5864, 2021.
Jonathan T Barron, Ben Mildenhall, Dor Verbin, Pratul P Srinivasan, and Peter Hedman. Mip-nerf 360: Unbounded anti-aliased neural radiance fields. In Proceedings of the IFEE/CVP Conference on Computer Vision and Pottern Recognition, pp. 5470 5479, 2022.
Amir Beck and Mare Teboulle. A fast iterative serrikagg--hresholldi algorithu for linear inverse problems. SIAM journal on imaging sciences, 2(1):183 2022 2009.
Beer. Bestimmung der absorption des rothen lichts in farbigen flüssigkeiten. Annalen der Physik, 162(5): 78-88, 1852.
David M. Blei, Alp Kucukelbir, and Jon D. McAuliffe. Variational inference: A review for statisticians. Journal of the American Stalistical Associalion, 1 112(518):859 877, 2017. doi: (0.1080/0162115992
Charles Blundell, Julien Cornebise, Koray Kavukeuoglu, and Daan Wierstra. Weight uncertainty in neural network. In International conference on machine learning, 1613 1622. PMLR, 2015.
Richard Castillo, Edward Castillo, Rudy Guerra, Valen E Johnson, Travis McPhail, Amit K Garg, and Thomas Guerrero. A framework for evaluation of deformable image registration spatial accuracy using large landmark point sets. Physics in Medicine & Biology, 54(())1849, 2009.
Guang-Hong Chen, Jie Tang, and Shuai Leng. Prior image constrained compressod sensing (pices): method accurately reconstruct dynamic images from highly undersampled projection data sets. Medical physics, 35(2):660 -63666363336666666 2008.
14

---

Published in Transactions on Machine Learning Research (09/2024)
Yinbo Chen and Xiaolong Wang. Transformers as meta-learners for implicit neural representations. In European Conference Computer Vision, pp. 170-187. Springer, 2022.
Hyungjin Chung, Byeongsu Sim, Dohoon Ryu, and Jong Chul Ye. Improving diffusion models for inverse problems using manifold constraints. Advances in Neural Information Processing Systems, 35:25683-25609 2022.
Francesco De Carlo, Doga Gürsoy, Daniel J Ching, K Joost Batenburg, Wolfgang Ludwig, Lucia Mancini, Federica Marone, Rajuund Mokso, Daniël M Polt, Jan Sijbers, et al. Tomobank: tomographic data repository for computational x-ray science. Mensurement Science and Technology, (933)0434 2018.
Alexander G. de G. Matthews, Jiri Hron, Mark Rowland, Richard E. Turner, and Zoubin Ghahramani. Gaussian process behaviour in wide deep neural networks. In International Conference on Learning Representations, 2018.
A. P. Dempster, N. M. Laird, and D. B. Rubin. Maximum likelihood from incomplete data via the em algorithm. Journal of the Royal Statistical Sociely: Series B (Methodological), 39(1):1-22 22, 1977. doi: https://doi.orr///
Henri Der Sarkissian, Felix Lucka, Maureen van Eijnatten, Giulia Colacicco, Sophia Bethany Coban, and Kees Joost Batenburg. A cone-beam x-ray computed tomography data collection designed for machine learning. Scientific data, 6(1):215, 2019.
Alexander Effland, Erich Kobler, Karl Kunisch, and Thomas Pock. Variational networks: An optimal control approach to early stopping variational methods for image restoration. Journal of mathemalical imaging and vision, 62:396-411 416, 2020.
Tommy Elfving, Per Christian Hansen, and Touraj Nikazad. Semi-convergence properties of kaczmarz's method. Inverse problems, 3055)0055007, 2014.
Alireza Fallah, Aryan Mokhtari, and Asuman Ozdaglar. On the convergence theory of gradient-based model- agnostic meta-learning algorithms. In Silvia Chiappa and Roberto Calandra (eds.), Proceedings of the Twenty Third International Conference on Artificial Intelligence and Statistics, volume 108 of Proceedings of Machine Learning Research, pp. 1082-1092. PMLR, 26 28 Aug 2020.
Chelsea Finn, Pieter Abbeel, and Sergey Levine. Model-agnostic meta-learning for fast adaptation of deep networks. In Proceedings of the 34th International Conference on Machine Learning Volume 70, 1CML'17, pp. 1126-1135. JogR.org, 2017.
Vincent Fortuin, Adrià Garriga-Alonso, Sebastian W. Ober, Florian Wenzel, Gunnar Ratsch, Richard E Turner, Mark van der Wilk, and Laurence Aitchison. Bayesian neural network priors revisited. In International Conference on Learning Representations, 2022.
Yarin Gal and Zoubin Ghahramani. Dropout as a baycsian approximation: Representing model uncertainty in deep learning. In international conference on machine learning, pp. 1050-1059. PMLR, 2016.
E Garduño, GT Herman, and R Davidi. Reconstruction from a few projections by mimiiiimiottion of the haar transform. Inverse problems, 27(5)055300, 2011.
Peter Gilbert. Iterative methods for the three-dimensionn reconstruction of an object from projections. Journal of theoretical biology, 36(1):105-11) 117, 1972.
Danicle Grattarola and Pierre Vandergheynst. Generalised implicit neural representations. Aduances in Neural Information Processing Systems, 35:30446 30458, 2022.
Yoscob Han and Jong Chul Ye. Framing u-net via deep convolutional framelets: Application sparso-view ct. IEFE transactions on medical imaging, 37(6):1418 -144998892,,,, 2018.
15

---

Published in Transactions on Machine Learning Research (09/2024)
Allard A Hendriksen, Dirk Schut, Willem Jan Palenstijn, Nicola Viganó, Jisoo Kim, Daniël M Pelt, Tristan Van Teeuwen, and K Joost Batenburg. Tomosipo: fast, flexible, and convenient 3d tomography for complex scanning geometries in python. Optics Express, 29(24):40494 40513, 2021.
Allard Adriaan Hendriksen, Daniël Maria Pelt, and K Joost Batenburg- Noisc2inverse: Self-supervised deep convolutional denoising for tomography. IEEE Transactions on Compulational Imaging, 6:1320 1335, 2020.
Gabor T Herman and Dewey Odhner. Performance evaluation of an iterative image reconstruction algorithm for positron emission tomography. IEEE transactions on medical imaging, 10(3):336 346, 1991.
Arthur Jacot, Franck Gabriel, and Clément Hongler. Neural tangent kernel: Convergence and generalization in neural networks. Advances in neural information processing systems, 31, 2018.
J.Dnesn. 3 walmuts. https://coomons..iiitiiit www.wedia. http:///wwo.....iiicaaoaa/oa//a/ www.wwsw.wl....aa.ae..e jpg. 2009, URL https:
7/ //creativecomoons wrre:///cccoooo.ooo//////ee////r httg///cennseerryaa//....000 Accossed: datc-of-access. Licensed under CC BY-SA
3.0. Modifications were made.
Daniil Kazantsev, Enyu Guo, AB Phillion, Philip J Withers, and Peter D Lec. Model-based iterative reconstruction using higher-order regularization of dynamic synchrotron data. Measurement Science and Technology, 289940444 2017.
Diederik P Kingma and Jimmy Ba. Adam: A method for stochastic optimization. ar preprint orXiv:1412.6600 69800, 2014.
Diederik P Kingma and Max Welling. Auto-cncoding variational bayes arXiv preprint ar1::131226614 2013.
Abhijit Kundu, Kyle Genova, Xiaoqi Yin, Alireza Fathi, Caroline Pantofaru, Leonidas Guibas, Andrea Tagliasacchi, Frank Dellaert, and Thomas Funkhouser. Panoptic neural fields: A semantic object-aware neural scene representation. In Proceedings of the IEEE/CVE Conference Computer Vision and Pattern Recognition, pp. 12871 12881, 2022.
Jacho Lec, Jihoon Tack, Namhoon Lee, and Jinwoo Shin. Mcta-learning sparse implicit neural representations. Advances in Neural Information Processing Systems, 14:11769-11788) 2021.
Lise Lotte Lene. Radiograf forbereder ct-scanning. https://commons:
After ort//Licenses/by-s//3..000 Ac- cessed: date-of-access. Licensed under CC BY-SA 3.0. Modifications were made.
flatting aww.anummmmmmmmmm rod. jpg. 2021. URL hhtps:: httpps://ceeatiieeecooo http://rrraaivecooooossssss
Zhengqi Li, Simon Niklaus, Noah Snavely, and Oliver Wang. Neural scene flow fields for space-time view synthesis of dynamic scenes. In Proceedings of the IEEE/CVP Conference on Compuler Vision and Pallern Recognition, pp. 6498 6508, 2021.
Jiaming Liu, Rushil Anirudh, Jayaraman J Thiagarajan, Stewart He, K Aditya Mohan, Ulugbek S Kamilov, and Hyojin Kim. Dolce: A model- based probabilistic diffusion framework for limited-angle et reconstruction. Proceedings of the IEEE/CVE International Conference on Computer Vision, pp. 10498 10508, 2023.
Yan Liu, Zhengrong Liang, Jianhua Ma, Hongbing Lu, Ke Wang, Hao Zhang, and William Moore. Total variation-stokes strategy for sparse-view x-ray et image reconstruction. IEFE transactions medical imaging, 33(3):7449700 2013.
Ziwei Liu, Ping Luo, Xiaogang Wang, and Xiaoou Tang. Deep learning face attributes in the wild. In Proceedings of the IEEE international conference on computer vision, pp. 3730 3738, 2015.
David JC MacKay. A practical bayesian framework for backpropagation networks. Neural computation, 4(3): 448 472, 1992.
Long Mai and Feng Liu. Motion-adjustable neural implicit video representation. In Proceedings of the IRFF/CVF Conference on Computer Vision and Pattern Recognition,, pp. 2022.
16

---

Published in Transactions on Machine Learning Research (09/2024)
Julien N. P. Martel, David B. Lindell, Connor Z. Lin, Eric R. Chan, Marco Monteiro, and Gordon Wetzstein. Acorn: Adaptive coordinate networks for neural scene representation. ACM Trans. Graph. (SIGGRAPH), 40(4), 2021.
Ricardo Martin-Brusc Brualla, Noha Radwan, Mehdi S. M. Sajjadi, Jonathan T. Barron, Alexey Dosovitskiy, and Daniel Duckworth. Nerf in the wild: Neural radiance fields for unconstrained photo collections. In Proceedings of the IEEE/CVF Conference on Compuler Vision and Pattern Recognition (CVPR), pp- June 2021.
Brendan McMahan, Eider Moore, Daniel Ramage, Seth Hampson, and Blaise Aguera y Arcas. Communicationn Efficient Learning of Deep Networks from Decentralized Data. In Aarti Singh and Jerry Zhu (eds.), Proceedings of the 20th International Conference on Artificial Intelligence and Statistics, volume 54 of Proceedings of Machine Learning Research, pp. 1273 1282. PMLR, 20 22 Apr 2017.
Ben Mildenhall, Pratul P Srinivasan, Matthew Tancik, Jonathan T Barron, Ravi Ramamoorthi, and Ren Ng. Nerf: Representing scenes as neural radiance fields for view synthesis. Communications of the ACM, 65(1): 99 106, 2021.
Amirali Molaei, Amirhossein Aminimehr, Armin Tavakoli, Amirhossein Kazerouni, Bobby Azad, Reza Azad, and Dorit Merhof. Implicit neural representation medical imaging: A comparative survey. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 2381 2391, 2023.
Thomas Müller, Alex Evans, Christoph Schied, and Alexander Keller. Instant neural graphics primitives with a multircsolution hash encoding. ACM Transactions on Graphics (ToG), 41(4):1-15, 2022.
Radford M Neal. Bayesian learning for neural networks, volume 118. Springer Science & Business Media, 2012.
Alex Nichol, Joshua Achiam, and John Schulman. On first-order meta-learning algorithms. arXiv preprint 1rXiv:1803.000 2018.
Shanzhou Niu, Yang Gao, Zhaoying Bian, Jing Huang, Wufan Chen, Gaohang Yu, Zhengrong Liang, and Jianhua Ma. Sparse-view x-ray et reconstruction via total generalized variation regularization. Physics in Medicine & Biology, (9(12)229 2014.
Albert W Reed, Hyojin Kim, Rushil Anirudh, K Aditya Mohan, Kyle Champley, Jingu Kang, and Suren Jayasuriya. Dynamic ct reconstruction from limited views with implicit neural representations and paramctric motion fields. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 2258 2268, 2021.
Gabriel Prieto Renieblas, Agustín Turrero Nogués, Alberto Muñoz González, Nieves Gómez-Leon, and Eduardo Guibelalde Del Castillo. Structural similarity index family for image quality assessment in radiological images. Journal of medical imaging, (03):0055501035000 2017.
Darius Rückert, Yuanhao Wang, Rui Li, Ramzi Idoughi, and Wolfgang Heidrich. Neat: Neural adaptive tomography. ACM Transactions Graphics (TOG), (14: 2022.
Liyue Shen, John Pauly, and Lei Xing. Nerp: implicit neural representation learning with prior embedding for sparsely sampled image reconstruction. IEEE Trensactions Neural Networks and Learning Systems, 2022.
Emil Y Sidky and Xiaochuan Pan. Image reconstruction in circular cone-beam computed tomography by constrained, total-variation minimization. Physics in Medicine & Biology, 53(1))44777, 2008.
Vincent Sitzmann, Julien Martel, Alexander Bergman, David Lindell, and Gordon Wetzstein. Implicit neural representations with periodic activation functions. Advances in Newral Information Processing Systems, 33: 7462-7473, 2020.
Bowen Song, Soo Min Kwon, Zecheng Zhang, Xinyu Hu, Qing Qu, and Liyue Shen. Solving inverse problems with latent diffusion models via hard data consistency. Xrxiv preprint ar orXi::2307.066 0080077........... 2023.
17

---

Published in Transactions on Machine Learning Research (09/2024)
Yang Song, Liyue Shen, Lei Xing, and Stefano Ermon. Solving inverse problems in medical imaging with score-based generative models. In International Conference on Learning Representations, 2022.
Nitish Srivastava, Geoffrey Hinton, Alex Krizhevsky, Πya Sutskever, and Ruslan Salakhutdinov. Dropout: a simple way to prevent neural networks from overfitting. The journal of machine learning research, 15(1): 1929-1958, 2014.
Yu Sun, Jiaming Liu, Mingyang Xie, Brendt Wohlberg, and Ulugbek S Kamilov. Coil: Coordinate-based internal learning for tomographic imaging. IEEE Transactions on Compulational Imaging, 111400-1412, 2021.
Matthew Tancik, Pratul Srinivasan, Ben Mildenhall, Sara Tridivich-Keill Nithin Raghavan, Utkarsh Singhal, Ravi Ramamoorthi, Jonathan Barron, and Ren Ng. Fourier fcatures let networks learn high frequency functions in low dimensional domains. Advances in Neural Informalion Processing Systems, 33:7537-777 7747, 2020.
Matthew Tancik, Ben Mildenhall, Terrance Wang, Divi Schmidt, Pratul P Srinivasan, Jonathan T Barron, and Ren Ng. Learned initializations for optimizing coordinate-based neural representations. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 2846 2866,,285,-- 2021.
Ayush Tewari, Justus Thies, Ben Mildenhall, Pratul Srinivasan, Edgar Tretschk, Wang Yifan, Christoph Lassner, Vincent Sitzmann, Ricardo Martin-Brualla, Stephen Lombardi, et al. Advances in neural rendering. In Computer Graphics Forum, volume 41, pp. 703 735. Wiley Online Library, 2022.
Wim Van Aarle, Willem Jan Palenstijn, Jeroen Cant, Eline Janssens, Folkert Bleichrodt, Andrei Dabravolski, Jan De Beenhouwer, K Joost Batenburg, and Jan Sijbers. Fast and flexible x-ray tomography using the astra toolbox. Optics express, 24(22):255 (22)):25129 25147, 2016.
FR Verdun, D Racine, JG Ott, MJ Tapiovaara, P Toroi, FO Bochud, WJH Veldkamp, A Schegerer, RW Bouwman, I Hernandez Giron, et al. Image quality in ct: From physical measurements to model observers. Physica Medicn, (8):8823 843, 2015.
Florian Wenzel, Kevin Roth, Bastiaan Veeling, Jakub Swiatkowski, Linh Tran, Stephan Mandt, Jasper Snoek, Tim Salimans, Rodolphe Jenatton, and Sebastian Nowozin. How good is the Bayes posterior in deep neural networks really? In Hal Daumé III and Aarti Singh (eds.), Proceedings of the 37th International Conference on Machine Learning, volume 119 of Proceedings of Machine Learning Research, pp. 10248 10259. PMLR, 13-18 18 Jul 2020.
Qing Wu, Lixuan Chen, Ce Wang, Hongjiang Wei, S Kevin Zhou, Jingyi Yu, and Yuyao Zhang, Unsupervised polychromatic ncural representation for et metal artifact reduction. Advances in Neural Information Processing Systems, 36:69605 69624, 2023a.
Qing Wu, Ruimin Feng, Hongjiang Wei, Jingyi Yu, and Yuyao Zhang, Self-supervised coordinate projection network for sparse-view computed tomography. IEEE Transactions on Computational Imaging, 2023b.
Weiwen Wu, Dianlin Hu, Chuang Niu, Hengyong Yu, Varut Vardhanabhuti, and Ge Wang. Drone: Dual- domain residual-based optimization network for sparse-vicw et reconstruction. IEEE Transactions on Medical Imaging, 40(11):3002-3011
2021.
Dejia Xu, Peihao Wang, Yifan Jiang, Zhiwen Fan, and Zhangyang Wang. Signal processing for implicit neural representations. Advances in Neural Information. Processing Systems, 35:13404-13411, 2022.
Kai Xu, Shiyu Lu, Bin Huang, Weiwen Wu, and Qiegen Liu. Stage-by-stage wavelet optimization refinement diffusion model for sparse-view et reconstruction. IEEE Transactions on Medical Imaging, 2024.
Siqi Ye, Saiprasad Ravishankar, Yong Long, and Jeffrey A Fessler. Spultra: Low-dose ct image roconstruction with joint statistical and learned image models. IEEE transactions on medical imaging, 39(3):729-77 -411, 2019.
18

---

Published in Transactions on Machine Learning Research (09/2024)
Guangming Zang, Mohamed Aly, Ramzi Idoughi, Peter Wonka, and Wolfgang Heidrich. Super-resolution and sparse view et reconstruction. In Proceedings of the Puropean Conference Computer Vision (ECCV), pp. 137- 153, 2018.
Guangming Zang, Ranzi Idoughi, Rui Li, Peter Wonka, and Wolfgang Heidrich. Intratomo: self-supervised learning- based tomography via sinogram synthesis and prediction. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 1960-1970, 2021.
Ruyi Zha, Yanhao Zhang, and Hongdong Li. Naf: Neural attenuation fields for sparse-view cbet reconstruction. 2022: 25th International
Medical Image Computing and Computer Assisted Intervention-MCCC..
Conference, Singapore, Seplember 18 22, 2022, Proceedings, Parl VI, pp. 442 452. Springer, 2022.
Hao Zhang, Jing Wang, Dong Zeng, Xi Tao, and Jianhua Ma. Rogularization strategics in statistical image reconstruction low-dose x-ray ct: review. Medical physics, 45(10)=e886 c907, 2018a.
Ruoqiao Zhang, Jlcan-Baptise Thibault, Charles Bouman, Ken D Sauer, and Jiang Hsich. Model-basod ilerative reconstruction for dual-energy x-ray of using a joint quadratic likelihood model. IEEE transactions on medical imaging, 33(1):117-134, 2013.
Zhicheng Zhang, Xiaokun Liang, Xu Dong, Yaoqin Xie, and Guohua Cao. sparse-view et reconstruction method basod on combination of densenet and deconvolution. IEEE transactions on medical imaging, (6):1407 1417, 2018b.
Bo Zhu, Jeremiah Liu, Stephen F Cauley, Bruce R Rosen, and Matthew S Rosen. Image reconstruction by domsin-transform manifold learning. Nature, 355(7697):487 492, 2018.
Junyi Zhu, Xingchen Ma, and Matthew Blaschko. Confidence-aware personalized federated learning via variational expectation maximization. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp. 24542-2455
June 2023.
A Appendix: Variational Expectation Maximization of Marginal Likelihood
To optimize the marginal likelihood p(yi.J σ), leverage a variational approximation of the network weights q(W).)) We start by rewriting the marginal likelihood as follows:
log (og = By((n.)) log P(y1.,, Wisjww (
(10)
(wi.) ((0::))((w1..)) Ni...ly1.... w.
p ply,,, Wi: ( +DKL
Egg(mn))) log
o)))
(11)
((w..))
ELBO(g([).)....))
Since directly maximizing the marginal likelihood is computationally infeasible, we instead maximize its variational lower bound, commonly known as the evidence lower bound (ELBO):
ELBO()(W).)).woo =Ey(w )) log P(U1:,, w1
(12)
(Wi)))
Ey log logp(yn. WiJ. w. www.llww::)lllbur........
(13)
The measurements are mutually independent and do not depend on the latent variables {w, a given network weights W1.J. Moreover, each network is trained solely on its corresponding measurement. Therefore, the log likelihood decomposcs into: Ey((w..)) log p(g:://w... w. = E E logp(y) w.,) Additionally, due the factorized variational approximation ((w..)) II ((w,) and the assumption of conditional independence p(wr.J /wo, σ) IT (.y,w, we obtain the following form of ELBO:
ELBO(d(W).))..oo w. Σ w logp(y, w,, D -Dila((;))lpll
(14)
19

---

Published in Transactions on Machine Learning Research (09/2024)
Since each network is trained separately, framework runs efficiently via parallel computing. The likelihood ((ي)(يn)) can be maximized by minimizing the squared error loss of the reconstruction (see Equation (2 Similarly to Zhuu et al. (2023), we adopt EM to maximize the ELBO. At the E-step, the latent variables {w, o are fixed and each network optimizes with respect to the loss function:
[(g(w))) = (ww AF (C) 1ı112 + Dx (1))alllll)) a))-
(15)
The expectation be estimated via MC sampling. When the variational approximations g Nug Ps) = 1...J J are formed, we can perform the M-step with the ((W1.)) being fixed. Simplilying
Equation 14
ELBO σ) - -
DEL ((la
(16)
1=1
By(w,) log logp(w,lw,o)
(17)
j=1
Σ
log diag(or) + (w 1=;)2. PDj. Pj. o _
(18)
1=1
where diag(o) represent diagonal matrix with the diagonal a. The above equation delivers a closed-form solution by setting the derivative to zero:
0 ELBO(W,,))
Σ w wu
w¹ = - Σ kij.
(19)
dw
ELBO r) &
Σo o (w uz)² - Pj = + 0º = Σ
(w" - 11j)² + Ps.
(20)
do
f=1
j=1
B Appendix: Details for Experiments
B.1 Noise Simulation for CT Reconstructions
We incorporate noise in our CT simulations by applying Poisson noise to measurements, following the approach described in (Hendriksen et al. 2020). According to Beer-Lambert's law (Beer 1852). the photon count detected /" function of the initial incoming photon count lo, the absorption factor 7 the object, and the measurement yo, expressed as:
1= To exp(7w))
(21)
The noisy measurement is modeled by introducing variability into the detected photon count. This is achieved by simulating the detected intensity as a Poisson-distributee variable relative to the scenario:
1o exp(rr) i (xiisson(oexx((1)))) y = -- To
(22)
(23)
In experiments, calibrate the noise level using two parameters: the average absorption y and the photon count Ig. For the WalnutCT datasct, we set 7 to 50% and To to 5000. For the LungCT dataset, we similarly set 7 to 50,, but with a higher photon count of In at 20000. This differential setting introduces varying degrees of noise to the measurements, as illustrated in Figure [I] The figure showcases how sparse mcasurements inherently challenge CT reconstruction, resulting blurred images due insufficient data. The addition of simulated noise further complicates this challenge by adding noise the already blurred images from sparse-view reconstructions.
20

---

Published in Transactions on Machine Learning Research (09/2024)
FBP w/o noise
SIRT w/o noise
FBP w noise
SIRT w noise
18.94
23.23
14.76
22
0.342
0.435
0.277
0.404
25. 11
28.28
19.53
27.25
MAAAA
The 0.346
0.517
0.709
0.610
Figure 11: Visual comparison of sparse CT reconstruction with and without simulated noise.
B.2 INRWild
We implement INRWild using the Siren network architecture (Sitzmann al.) 2020). Specifically, the static component, GΦ. is represented using standard 8-layer Siren network. In contrast, each of the transient parts, Hm,; is characterized by more compact 4-layer Siren network following the design from NeRFWild Alartin-Brualla et al.) 2021). Each transient part is linked to unique one-hot vector, which is embedded into 16-dimensional transient feature by. This feature, alongside intermediate features G (C) from the static network, serves input for transient modeling. The transient feature by is optimized during joint training.
The optimization process ensures that the static and transient components are jointly optimized to ensure distinct representations, as articulated in Equation [3] A visual representation of the INRWild structure is
provided in Figure 12
G
(C)
Static Density
Positions
MLP
Hosy
g. (C)
MLP
Transient Density
Transient Feature
by
Figure 12: Schematic depiction of INRWild: A tailored version adapted from NeRFWild (Martin Brualla
ct al. 2021 for joint CT reconstruction.
B.3 INR-based Network Configuration
In this study, all INR-based methods employ the Siren network (Sitzmann et al. 2020) their core architecture. For each experimental setup, we consistently use a Siren network with specific dimensions: a depth of 8 and width 128 for the WalnutCT experiments, and a depth experiments. Position encoding is implemented using Fourier feature embedding (Tancik et al.) 22020), with an embedding dimension 512 for LungCT and 4DCT, and 256 for CalnutCT. Consistently, this embedded position is input to the INRs to facilitate density prediction.
with width 256 for the LungCT and 4DCT
21

---

Published in Transactions on Machine Learning Research (09/2024)
B.4 Configuration of All Methods
Individual Reconstruction Methods. For 2D experiments, we employ the FBP method, and for 3D experiments, its counterpart, FDK. GPU-accelerated operations, CUDA for 2D and FDK_CUD, for 3D, are sourced from the Astra-Toolbos (Van Aarle el al. 2016). The iterative method SIRT, specifically the SIRT_CUDA operation, also from the Astra- Toolbox, is configured with 5,000 iterations. INR-based methods are set to operate for 30,000 itcrations. The itcration counts for SIRT and INR-based methods are determinod through preliminary lests on a datasel subset to ensure optimal performance.
Hyperparameters. We tune the hyperparameters of our method and baselines in the inter-object scenario and subsequently apply to all experiments. In particular, for the federated averaging approach FedAvg (Kundu et al. 2022). we average individual reconstructions every 100 iterations, amounting to a total of 200 averaging iterations, followed by 10,000 individual adaptation iterations using the learned meta initialization. For the meta-learning technique MAML (Tancik et al. 2021), which is rooted in the MAML update policy (Finn et al. 2017), ; we designate meta learning phase spanning 10,000 iterations (10 inner steps and 1000 outer iterations) across all measurements. This is succeeded by 20,000 individual adaptation iterations utilizing the learned meta initialization. Our INR-Bayes undergocs 300 EM loops. For cach loop, the E-step iterates 100 times to update the posterior approximation. All INR-based models utilize the Adam optimizer Kingma. & Ba (2014) with the first moment 0.9 and the second moment 0.999. The learning rate is set to 1 x 10 5, For our method, the additional hyperparameter B for the KL divergence term is determined as 1 x 10- 14 for WalnutCT and 1 10-16 for LungCT, 4DCT.
Final Reconstructrion. After training, ours INR-Bayes yields a parameter distribution. Common methods for producing the final result based on this distribution include sampling the network paramcters multiple times and averaging the network outputs, using the mode of the distribution as the network parameters to produce the result (i.e., maximum a posteriori). In our work, we choose the latter approach, as it empirically yields better performance. For the baseline methods, we use the final trained model to produce the final reconstruction.
C
Appendix:
Dataset Details
4DCT. This dataset Castillo et al. (2009), sized 10 x 136 x 512 x 512, contains 136 CT image slices captured across 10 respiratory phases of one patient. The main variations across these phases are due to respiratory movements, such as the lungs' expansion and contraction.
LungCT. Comprising CT scans from 96 patients Antonelli to 636 x 512 x 512. Despite inherent similarities representing human lungs, individual scan features vary significantly. Slices wit a volume show a consistent pattern, yet fewer stationary features are shared between them. The dataset comprises scans both with and without tumors. For our experiments, we randomly selected patients and images without distinguishing between those containing tumors and those without, aiming for a diverse representation of lung CT images.
Antooels et al. 2022 its volumes range from x12 x12
WalnutCT. This dataset Der Sarkissian et al. (2019) features 42 CT volumes of walnuts, each sized 500 501 x 501.
AluminiumCT. Comprising 25 CT scans of an aluminium alloy tested across various fatigue cycles De Carlo et al. (2018), each original scan measures 2160 x 2560 x 2560. For computational feasibility, we preprocess these by averaging every five columns in each detector row, reducing the resolution to 2160 x 512 x 512. This dataset allows us to explore the structural integrity and detect minute defects within the material under different stress conditions, simulating practical industrial applications.
CelebA. This dataset Li et al. (2021) consists of 202,599 celebrity face images of dimension 3 x 218 x 178.
22

---

Published in Transactions on Machine Learning Research (09/2024)
FBP
SIRT
SingleINR
FedAvg
MAML
INR-Bayes
GT
23.77
30.18
29.99
24.99
30.13
31.47
0.578
0.778
0.774
0.557
0.773
0.855
23.41
31.01
30.97
24.43
30.66
31.38
0.653
0.835
0.821
0.568
0.823
0.837
Figure 13: Visual comparison on human faces results. PSNR values are on the top left, with SSIM values on the bottom left.
D Appendix: Extra Results
D.1
3D Joint Reconstruction
To assess the real-world viability of our method, we conduct evaluations in a 3D cone-beam CT context, which more closely aligns with practical scenarios. We choose CT volumes from different patients of size 1283, ensuring they represent analogous regions the human body. The projections are simulated with 40 angles spanning a full 360 rotation.
We conduct experiments on groups of joint reconstructions, with each group jointly reconstructing 10 different patients' CT volumes, each sized 1283. Table displays the results. Consistent with the findings from 2D CT experiments, our approach surpasses other comparative methods, substantiating its practical relevance. MAML displays slightly inferior performance compared to SingleINR. A potential rationale for this could be that, given the augmented data volume, meta-learning might necessitate extended meta-learning iterations to glean a meaningful representation.
FDK
SIRT
SingleINN
FedAvg
MAML
INR-Bayes
PSNR 19.40 =0.02 24.91
40.45
33.99 40.42
30.30 40.27
33.67 40.55
34.19 +0.38
SSIM 0.550 40.005 0.650 40.007 0.932 40.007
0.862 40.004
0.932 #0.009
0.945 +00004
Table 4: Results from 3D cone-beam CT reconstruction. The highost average PSNR/SSIM values that are
statistically significant are highlighted in bold.
D.2 Joint Reconstruction on Human Faces
To offer deeper insight, visualize the learned priors across different joint reconstruction techniques. Figure 14 illustrates INRWild's extraction of 'static' and 'transient' components from varying faces. Notably, INRWild captures a generalized "face" static component. However, due to the significant disparities among face images, this generalized extraction does not significantly enhance individual reconstructions. Figure 15 showcases the learned priors from FedAvg, MAML, and our approach INR-Bayes, Both FedAvg and IMR-Bayes succeed deriving an interpretable prior. However, the face-like meta initialization does not directly benefit reconstruction in the case of FedAvg. In contrast, MAML struggles to capture a face-like prior during its preliminary phase, but obtains final reconstructions better than FedAvg. Our INR-Bayes usually
23

---

Published in Transactions on Machine Learning Research (09/2024)
reconstructs images closest to the ground truth. It forms prior that contains color lumps similar to MAML well as a face-like feature like FedAvg.
GT
Learned Static
Transient
Composite
Node 1
266
15.70
Node 2
190
Same
21.33
Node 3
0.42
Figure 14: Learned static and transient parts of INRWild human faces of CelebA dataset.
24

---

Published in Transactions on Machine Learning Research (09/2024)
GT
FedAvg
MAML
INR-Bayes
Node 1
0,866
0,821
0.832
27.41
30.22
31 ..99
Node
0.760
0.826
26.69
32.73
33.63
Node
0.675
0.863
0.915
Learned Prior
Figure 15: Visualization of the learned prior of joint reconstruction methods on the faces of the CelebA datasot.
D.3 Inter-object Joint Reconstruction
In this section, we provide visual comparison of inter-object joint reconstruction using the WalnutT., As depicted in Figure 16 approach exhibits the best PSNR and SSTM values, and delivers result. that aligns better with the ground truth.
SIRT
Single INR
INRWild
FedAvg
MAML
INR-Bayes
GT
74.12
36.74
22.35
$0.35
$7.00
S7 88
0.570
0.956
0.784
0.907
0.962
0.968
23.75
31.72
17.65
29.48
31.97
33.44
0.582
0.906
0.687
0.328
0.917
0.937
Figure 16: Visual comparison for inter-object walnut joint reconstruction. Falarged highlighted in red insets. PSNR values are the top left, with SSIM values on the bottom left.
25

---

Published in Transactions on Machine Learning Research (09/2024)
D.4 Intra-object Joint Reconstruction
In addition to the experiments represented by Figure 10 evaluated how the number of nodes influences the performance of varions methods, in the intra-object joint reconstruction experiments. As shown in Figure [7 our method consistently delivers superior performance compared to other methods across a range of node counts. MAML shows strong results when the node count is between and 25, but experiences a decline performance, eventually matching that of FedAvg when the node count reaches 40. This drop indicates that MAML might struggle capture the shared features when many nodes are participating in the joint reconstruction. Contrary to the inter-object configuration (as illustrated in Figure 10), in this experiment our method does not benefit. from increase in the number of joint nodes. This lack of improvement could be attributed to the possibility that the statistical regularitios are sufficient when observing just few scanning slices in the intra-object setting.
36
INR-Bayes
30
MAML
FedAvg
28-
Sing lagl
SIRT
26
10
15
20
25
30
35
40
Number Nodes
Figure 17: Performance comparison with varying node numbers in intra-object LungCT setup. Individual reconstruction methods are presented as references. Performance stability across all methods with incrcasing node numbers suggests that even a limited number of intra-object slices adequately capture statistical trends.
We also present visualizations of learned priors from various joint reconstruction methods applied in intra- object experiments on LungCT dataset. Figure I8 depicts the learned static and transient components of INRWild. Notably, in scenarios where images markedly vary from one anothor, extracting static components still seems feasible but not necessarily beneficial to the reconstruction process, since the static component. does not constitute a significant portion of the overall representation.
Conversely, when observing other joint reconstruction methods in Figure 19 it is evident that INR-Bayes, MAML and FedAvg ascertain reasonable latent meta representation. Despite variations in images, the intrinsic consistency stemming from the same patient results in a discernible and coherent trend. This inherent trend is adeptly captured by these joint reconstruction methodologies.
D.5 Joint Reconstruction across Temporal Phases in 4DCT
Figure 21 reveals that INRWild effectively distinguishes static and transient components in the 4DCT setting. Compared to other settings, INRWild's performance on 4DCT is closer to other INR-based methods (see Table 1. However, it still lags behind SingleINR, underscoring the limited utility of static-transient tactics in enhancing individual reconstruction quality. Figure 22 demonstrates that other joint reconstruction methods also proficiently disentangle the inherent prior. Interestingly, in these contexts, FedAvg proves more beneficial than MAML, while our proposed method continues to surpass all methods in this scenario.
26

---

Published in Transactions on Machine Learning Research (09/2024)
GT
Learned Static
Learned Transient
Composite
27.45
Node 1
0.690
27.85
Node 3
Same
0.677
27.74
Node 5
0.709
Figure 18: Learned static and transient parts of INRWild on LungCT dataset on the same patient.
D.6 Joint Reconstructions with Varied Object and Slice Configurations
To understand how different joint reconstruction configurations impact reconstruction quality, we examine four distinct setups: inter-object (10 objects, each contributing one slice), intra-object (one object contributing ten slices), and two intermediate configurations (two objects with five slices each, and five objects with two slices cach). We conducted ten sets of experiments, with each set involving the joint reconstruction of the same len slices across these configurations.
Table presents the average PSNR and SSIM values obtained from these ten experimental groups. The results indicate that while the PSNR and SSIM values are generally consistent across different configurations, the intermediate configurations slightly outperform the others in terms of average metrics.
Figure 23 illustrates the learned priors from one exemplary set of reconstructions. Visually, the distinctions among the different configurations are minimal. The intra-object configuration (1x10) effectively captures the internal variations within single object, aligning with our observation that few intra-object slices are sufficient to model internal object changes. On the other hand, the 2x5 and 5x2 configurations, along with the inter-object (10x1) setup, depict priors that appear more distinct from the target slice, reflecting the influence of multiple objects the joint reconstruction process. Notably, the 2x5 and 5x2 configurations exhibit slightly higher PSNR and SSIM values, suggesting that an optimal balance between similarity and diversity among jointly reconstructed objects enhance the reconstruction quality of specific target.
Config (object x slices)
1 x 10
2 x 5
5 x 2
10 x
PSNR
35.56 #1.03
35.76 11.08
35.66 11.07
35.59 11.02
SSIM
0.876 10.025 0.877 40.025 0.878 22.025 0.876 20.025
Table 5: Results from joint Reconstructions with varied object and slice configurations.
27

---

Published in Transactions on Machine Learning Research (09/2024)
GT
FedAvg
MAML
INR- Bay-Bayes
32.45
35.78
36.43
Node 1
0.827
0.879
0.897
38.47
40.44
41.17
Node 3
0.932
0.950
0.958
32.07
33.36
34.74
Node 5
0.828
0.841
0.874
Learned Prior
Figure 19: Visualization of the learned prior of joint reconstruction methods on the same patient of LungCT dataset.
SingleINR
INRWild
FedAvg
INR-Bayes
33.40
Phase
0.729
0.89
0.888
0.880
0.887
0.892
27.78
33.1
33.88
34.09
89.61
34.21
Phase
0380
0.883
0.893
Figure 20: Visual comparison for joint reconstruction across 4DCT temporal phases. Enlarged areas are highlighted in red insets. PSNR values are on the top left, with SSTM values on the bottom left.
D.7 Computation Cost Analysis
In Table we present a comparative analysis of the computational costs associated with different reconstruction methods. The experiment setting is aligned with the inter-object configuration on CalnutCT dataset Table T These asscssments are performed under the same conditions with 40GB A100 GPU, ensure consistency our evaluation. Each method undergoes 30,000 iterations and employes an identical Siren network architecture, characterized by depth of 8, width of 128, and positional embedding dimension of 256.
28

---

Published in Transactions on Machine Learning Research (09/2024)
GT
Learned Static
Learned Transient
Composite
32.58
Phase 1
0.820
33.08
Phase 3
-Same-
0.822
33.12
Phase 5
0.82
Figure 21: Learned static and transient parts of INRWild 4DCT dataset.
SingleINR FedAvg MAML INR-Bayes
GPU Memory (MiB)
3852
3972
3954
3990
Time (mins)
17.2
17.2
17.2
22.7
Table 6: Comparison of average computation cost per node on joint reconstruction of 10 nodes. The reconstructed image size is 501 x 501. For reference, FBP requires 206 MiB of GPU memory and completes in less than 0.1 seconds, whereas SIRT utilizes 154 MiB of GPU memory and takes 0.8 seconds.
As indicated in the Table 6 the GPU memory usage across all INR-based methods was relatively similar. SingleINR, FedAvg and MAML exhibit the same shortest computation time. Our method, INR-Bayes, shows a slightly increased computation time, approximately minutes longer than the others. This less than 15% increase over SingleINR, FedAvg and MAML time attributed to the MC sampling procedure (~ 9%) and additional KL divergence term (~ 6%) in INR-Bayes. However, considering the enhanced reconstruction quality and robustness achieved, this additional time investment can be justified.
Impact of Network Size on INRBayes. To understand the computation efficiency and performance trade-off, we further investigate the reconstruction quality of our method with varying configurations of the network size. As shown in Table smaller networks lead to reduced performance but also lower computational requirements, presenting a practical tradc-off between efficiency and quality. It worth noting that even with the smallest network, our method largely outperforms SIRT. While the reconstruction time using the smallest network is reduced to 10 minutes, which could be affordable in the most practical settings.
29

---

Published in Transactions on Machine Learning Research (09/2024)
GT
FedAvg
MAML
INR- -NR-Bayes
34.94
35.40
35.70
Phase 1
0.880
0.905
0.910
35.30
32.71
35.85
Phase 3
0.904
0.877
0.914
34.97
34.42
35.40
Phase 5
0.899
0.898
0.91
Learned Prior
Figure 22: Visualization of learned prior of joint reconstruction methods on 4DCT datasct.
1x10
2x5
5x2
10x1
GT
38.71
40.36
40.09
39.66
INR-Bayes
0.935
0.933
0.938
0.934
Learned Prior
Figure 23: The reconstructions and learned priors with varied object and slice configurations.
Depth Embedding Size
PSNR
SSIM
Memory (MiB) Time (miin))
256
35.45 #0.35 0.944
20.003
3990
22.7
8
128
33.20
40.49 0.918
20.006
3900
22.5
6
256
33.75 #0.39 0.930
$0.005
3468
18.3
256
30.55 #0.37 0.906 D0.006
2980
10.6
Table 7: Impact of the network size on computation cost and performance of our method. For comparison, SIRT achieves PSNR of 23.42 +0.21 and SSIM of 0.445 =0.004. The experiment setting is inter-object CalnutCT with 30,000 total iterations.
30

---

Published in Transactions on Machine Learning Research (09/2024)
D.8 Comparison with Nerp
The method Nerp; introduced by Shen el al. (2022), initially trains INR network using high-fidelitty data through regression. This pre-trained network is subsequently utilized to initialize the reconstruction of a new object with sparse measurements. A notable drawback of this method its dependence on the new objects' representations being highly similar to that of the high-fidelity training object. When this similarity is absent, the initial training could hinder rather than help the reconstruction process, potentially yielding worse results than even random initialization.
To demonstrate this, carried out experiments on the 4DCT dataset with two different setups for Nerp. In the 'Match' configuration, Nerp is provided with the ground truth of one phase at specific slice and tasked to reconstruct the remaining nine phases at that slice. In contrast, the 'Unmatch' configuration uses the ground truth from a random slice. Our INR-Bayes approach, on the other hand, performs a simultancous recoustruction of all nine phases without any access to ground truth images.
As Table S jillustrates, the performance of Nerp is conditional, excelling in PSNR when ground truth data is matched but faltering otherwise. While operating without access to additional information, IMR-Bayes achieves the best performance in SSIM. Given the practical challenges in obtaining matched ground-truth data for unscanned objects, our method exhibits greater utility and applicability.
SingleINR Nerp match Nerp unmatch INR-Bayes
PSNR 33.69 10.06 35.33 20.10 32.83 40.07
34.31 10.07
SSIM 0.883 =0.002 0.889 10.001
0.849 40.02
0.901 +.0001
Table Comparison of Nerp performance scenarios where the prior image either matches or does not match the target in the 4DCT dataset. The highest. average PSNR/SSIM values that statistically significant are highlighted in bold.
D.9 Applying to Unseen Data using Different Learned Priors
To investigate the influence of varying priors on the reconstruction quality for new, unseen patients, we conducted an additional experiment with our INR-Bayes. We selected 10 sets of priors, each derived from a group of 10 different paticnts. These priors are then employed to guide the reconstruction of the same unscen patient. Figure 25 showcases the reconstructed images and their corresponding priors, represented by an INR that is parameterized with the mean of the prior distribution. The accompanying PSNR and SSTM values, indicated at the top left and bottom left of each image, demonstrate modest deviation across different priors. Notably, no model collapse occurs despite the obvious visual difference in the prior means. This observation suggests that our method is stable and can adaptively extract useful information from various priors when applied to unseen data.
It important to clarify that the prior of our method, as depicted in Figure 25 uses the mean of the prior distribution to parameterize an INR. However, this representation is an incomplete portrayal of the prior distribution's full characteristics, the latent variables include mean and variance estimations. The variance associated with our method's estimates can contribute to the adaplive and elfective utilization of the prior distribution. This aspect of our model underscores its capability to leverage the entire prior distribution for stable performance.
D.10 Reconstruction Uncertainty
We employ Bayesian framework to identify common patterns among jointly reconstructed objects. While the primary focus of this study is not on uncertainty quantification (Gal & Ghahramani 2016 Angelopoulos el al.) 2022), we demonstrate how INR-Bayes can facilitate this aspect within CT reconstructions. During training, each node approximates the posterior distribution of its weights as a Gaussian distribution. This approach allows sample the network parameters multiple times after training to quantify uncertainty. To illustrate this, we sample the network parameters ten times to generate ten distinct reconstructions in an intra-object.
31

---

Published in Transactions on Machine Learning Research (09/2024)
target image
measurement
SingleINR
33.55
0.866
prior image
(target-prior)-2
Nerp
33.24
35.90
prior matches
0.924
0.908
17.51
33.00
prior unmatches
0.683
0.886
Figure 24: Results of Nerp with matched prior image and unmatched prior image. PSNR on the upper left corner and SSIM on the lower left corner are calculated with respect to the target image.
setup using the LangCT dataset. We then compute the mean and variance of these reconstructions to show the expected reconstruction and characterize the associated uncertainty in Figure 26
32

---

Published in Transactions on Machine Learning Research (09/2024)
Prior
30.94
30.70
30.72
30.90
30.89
INR- Ba-ea
Prior
30.83
30.85
30.76
30,68
30.85
INR-Bayes
0.785
0.784
0.785
Figure 25: Reconstruction of the same unseen patient using different priors learned from various patient groups. The PSNR and SSIM values are presented on the top left and bottom left of each image, respectively, illustrating our method's robustness across diffcrent priors.
GT
Mean
Variance
Figure 26: Mean and variance of 10 reconstructions by INR-Bayes.
33


Reviews:
[{"review_id": "c4dkrQG0Pd", "reviewer": "Reviewer_k9NV", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "+ The paper is clearly written. Necessary and detailed background information is provided in the paper, so that the reader can understand the idea easily.\n+ The paper contains detailed comparisons between INR-Bayes and baseline algorithms.\n- Checking Table 1 and Figure 3 we can see SingleINR and MAML are the strongest baselines. More comparisons can be done to prove INR-Bayes is better than these two.", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "Yes", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "In this submission, the authors proposed INR-Bayes, a Bayesian-based joint reconstruction method for CT imaging. They proposed to learn neural representation of multiple object jointly, and the neural network weights are sampled from a multivariate Gaussian. To optimize the likelihood, they maximize the evidence lower bound using the EM algorithm. The authors demonstrated their method’s effectiveness both qualitatively and quantitatively.", "claims_and_evidence_yn": "Yes", "claims_explanation": "", "audience_yn": "Yes", "audience_explanation": "", "requested_changes": "* SingleINR and MAML are strong baselines. It would be great if we have more comparisons to them and can clearly demonstrate INR-Bayes's advantages over them.\n* More comparisons between inter and intra-object joint reconstructions. Is it better to optimize on different views of a single object, or different views of multiple objects? Note that we can have more slices when optimizing on multiple objects. What would be the difference of the learned priors?", "broader_impact_concerns": "No ethical concerns.", "additional_comments": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "mMIwRhed40", "reviewer": "Reviewer_2CxD", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "Strengths:\n- Efficient and fast reconstruction of several objects from CT measurements is a relevant problem.\n- The proposed method is novel.\n- Results show improvements compared to existing methods.\n\nWeaknesses:\n- Extensive use of jargon makes presentation less accessible to first-time readers\n- Notation and presentation of main contributions is somewhat hand-wavy\n- Certain claims could be made clearer or reworded.\n\nI will expand on my comments and I am looking forward to discussing with the authors to clarify my confusion.", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "Yes", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "The submission proposes a novel Bayesian approach for the joint reconstruction of several objects from sparse-view CT measurements. The proposed method consists of training implicit neural representations (INRs) concurrently---one per object being reconstructed---by coupling them with a latent prior that should encode common information across objects. Experiments and ablation tests are carried out to compare the proposed method with existing INR-based alternatives on 4 different datasets coming from different domains.", "claims_and_evidence_yn": "Yes", "claims_explanation": "", "audience_yn": "Yes", "audience_explanation": "", "requested_changes": "**Disclaimer on use of language models**\n\nI use GPTZero to check all abstracts of submissions I review. For this submission, the abstract was reported as 100% AI-generated. I would urge authors to review TMLR's guidelines on the disclosure of the use of language models, available at https://jmlr.org/tmlr/faq.html .\n\n---\n\n**Abstract**\n\nUse of jargon makes the text unintuitive to people not familiar with the topic. In particular:\n\n* \"we introduce a novel ... the inter-object relationships\" is hand-wavy before reading the paper.\n* \"meta-initialization\", \"dynamic reference\", \"individual reconstruction fidelity\", \"resistance to overfitting\", and \"generalizability\" are all used without definition. These are very broad terms with different meaning and possible interpretations.\n\nI would suggest to smooth the claim: \n> This underscores a notable advancement in CT reconstruction methods.\n\n---\n\n**Introduction**\n\n> posing a complex inverse problem.\n\n\"complex\" may be mistook here for \"complex number\" instead of \"complicated\".\n\n> However, this sparsity in measurements complicates the reconstruction process, making it an ill-posed inverse problem.\n\nIll-posedness of the problem does not stem only from the use of Sparse-view CT but also other factors such as number of detectors, geometry of the scanner, and the presence of noise.\n\n> ... to numerous inverse problems.\n\nIt may be helpful to include a couple examples.\n\n> Supervised learning ... Alternatively, Song et al ...\n\nIt may be useful to rephrase this paragraph to stress that supervised learning and diffusion models are not the only two perspectives on CT reconstruction. Also, several works have specialized the fundamental ideas of Song et al. to CT applications, and they may be worth mentioning here for a fair representation of the research landscape.\n\n- Typo in \"there are also some studies leverage\".\n\n> On a different tangent ...\n\nThis makes it sound as if what comes next is only indirectly related to the problem of CT reconstruction, but the paragraph introduces INR-based methods, which are at the core of the contributions of the submission.\n\n> Thanks to the continuous representation nature of INRs ...\n\nI am not sure I follow how continuity leads to better reconstruction results, there may be other factors in support of INRs?\n\n> Given INRs' ... and the known advantages of leveraging auxiliary information ...\n\nIt is unclear what \"auxiliary information\" and \"advantages\" refer to here.\n\n> ... among different objects borne in the INR networks' weights ... cannot fully capitalize on the available statistical regularities ...\n\nIt is unclear to me what this sentence means. What does it mean for an object to be borne in the weights of a network? How can one measure if a method is \"fully capitalizing\" compared to \"partially capitalizing\"?\n\n---\n\n**Related work**\n\n> ... with a comprehensive understanding on INR and NeRF available in the survey (Tewari et al., 2022).\n\nINRs are at the core of the contributions of the submission, but they are not formally introduced. A more careful introduction of these ideas is necessary to make the submission accessible to readers coming from the broader CT reconstruction field who may not necessarily know about INRs.\n\nFurthermore, the cited survey \"[does] not focus on neural rendering methods that reason mostly in 2D screen space\". It is my understanding that the objects studied in this submission are 2D images. Finally, \"the authors [of the survey] do not claim completeness of the report and highly recommend the reader to study the cited works for in-depth details\", so I would suggest introducing the core ideas directly in the main text of the submission and including more foundational introductory references for interested readers who might want to learn the details.\n\n> Coordinates-based ... have transitioned from traditional discrete representations ... detailing issues.\n\nIt may be helpful to expand on this sentence. In its current form, it is unclear what \"discrete representations\" are (i.e., pixels on a grid). \"have transitioned\" is a very broad claim, and traditional MLPs are still prominent in current research. Finally, without further explanation, it is unclear what \"addressing high-frequency function detailing issues\" means.\n\nOnward, this paragraph describes concepts and ideas related to scene understanding (e.g., \"viewing angles\", \"transmittance effects\", \"ray-tracing\", \"video-representation\", ...). It is unclear to me how these topics are relevant to set the stage for the presentation of the main contributions of the submission, and, to someone without specific scene understanding knowledge, they are confusing.\n\n---\n\n**Problem Statement and Preliminaries**\n\n> ... where $\\epsilon$ accounts for the associated measurement noise.\n\nIt may be helpful to specify that $\\epsilon$ is a stochastic quantity. Could the authors expand on what kind of noise they consider? In this form, it seems that $\\epsilon$ is independent additive noise (maybe Gaussian?), but later in Appendix B.1, Eq. (22) introduces Poisson noise, which is not additive.\n\n> [the INR designed for CT reconstruction] maps ... to its intensity in a continuous three-dimensional space.\n\nIt is unclear whether the \"continuous three-dimensional space\" is the input or output domain of the INR. Intensity is a scalar value.\n\n> ... and measurement matrices $A_{1:J}$.\n\nThe joint reconstruction problem is introduced with different measurement matrices. Could the authors expand on whether in their experiments all objects being reconstructed have the same measurement matrix $A$, or how it differs across objects.\n\n> ... can provide a more principled approach.\n\nCould the authors expand on in what sense is including a dynamic prior shared across nodes more principled?\n\n> ... and corresponding transient feature $b_j$.\n\nCould the authors expand on whether these features are learned, and if so how?\n\n> Here, $\\mathcal{G}_{\\phi}^{r}(C)$ ... and $\\mathcal{G}_{\\phi}^{\\r}(C)$ ...\n\nThese symbols are neither defined in the main text, nor in Appendix B.2 .\n\n> ... characterized ... characteristics.\n\nNote the repetition. What characteristics of $\\phi$ are being referred to here?\n\n* Eq. (4):\n    - It may be helpful to use the same notation $w^{(k)}$ and $\\theta^{(t)}$ instead of $\\theta^t$.\n    - Is there a missing transpose in Eq. (4) between the gradient and the Jacobian matrix?\n\n---\n\n**A Novel Bayesian Framework for Joint Reconstruction**\n\n> ... all representation substantially overlap?\n\nCould the author expand on what it means for representations to overlap.\n\n* Role of transient features in proposed method\n\nIt was unclear to me whether transient features are used in the proposed method, and, if so, how they are learned within the Bayesian framework.\n\n> While this posterior enables various forms of deductive reasoning.\n\nI am not sure I follow the meaning of this sentence in the context of the paragraph.\n\n> ... that aims at maximizing the marginal likelihood $p(y_{1:J} \\mid \\omega, \\sigma)$.\n\nWhy is this term called marginal likelihood if it is conditional on the latent variables?\n\n> ... parameterizing the variance $\\sigma_j$ ...\n\nTypo in parametrizing. Also, should this be $\\rho_j$ instead of $\\sigma_j$?\n\n* Eq. (5) and (7)\n\nI would suggest to maintain consistent notation of the inputs of the ELBO function across equations.\n\n* Questions on proposed method\n\nIt may be useful to remark that the latent space is as big as the nodes. It is common for latent spaces to have a dimension smaller than the object of interest. Here, the latent space keeps track of the mean and variance of the nodes, so its dimension is fixed.\n\nCould the authors expand on whether the proposed method can be extended to latent spaces smaller than the nodes? For example, what would happen if the latent space were as large as the images being reconstructed? Would this retrieve a similar approach to INRWild?\n\nMaybe, it could be interesting to include some example images generated from the learned prior, since the prior is an INR itself.\n\nCould the authors expand on the difference between the proposed method and the federated learning approach FedAvg? The latent mean $\\omega$ can be interpreted as a meta network, so the main advantage is in considering the variance as well?\n\nSimilarly, all nodes are assumed to be the same size, but this assumption is not stated clearly in the text. I would suggest including this remark. Could the proposed method entail nodes of different size? What would the main hurdles to achieve that be?\n\nFinally, could the authors expand on the choice of using the posterior expectations instead of sampling? An advantage of Bayesian methods is to include uncertainty quantification out-of-the box, and it would be interesting to compare with diffusion-based methods.\n\n---\n\n**Experiments**\n\n* Missing reference for SIRT:\n\nGilbert, Peter. \"Iterative methods for the three-dimensional reconstruction of an object from projections.\" Journal of theoretical biology 36.1 (1972): 105-117.\n\n* Typo in \"for the rest experiments\"\n\n* CelebA experiments:\n\nIt is unusual to see the CelebA dataset with a CT forward model. Have the authors considered showcasing their proposed method for a different, more common task on the CelebA dataset, such as image denoising or inpainting? This may better place the proposed method in comparison with existing denoising techniques that also do not rely on the availability of large datasets.\n\n* Clearly state if results are on noiseless reconstruction\n\nCT reconstruction is presented as a noisy inverse problem. However, Table 1 shows results for noiseless measurements. This should be stated clearly before presenting results.\n\nCould the authors expand on their motivation for comparing on all settings for noiseless measurements and only on the inter-object setup for noisy measurements?\n\n> ... overfitting when applied to limited data.\n\nCould the authors expand on this claim? Iterative reconstruction methods do not require a trained predictor, so what does \"limited data\" refer to here? What kind of overfitting is being referred here since measurements are noiseless at this point in the submission. In a noiseless setting, this may be an issue of stability and convergence of the optimization process rather than overfitting?\n\n* It is never specified which method the curves in Figure 7 are generated with. Similarly to my confusion above, Figure 7 is used as evidence of overfitting in the noisy setting, but similar claims are also made with Figure 4 in the noiseless setting.\n\n> ... we select 5 consecutive slices from new objects, choosing slices from a similar location the prior has been trained.\n\nIt is not specified how many times this process was repeated to obtain confidence intervals. What does \"similar location\" mean? Is there a particular tolerance that was used to include slices from different objects?\n\n* Broader applicability\n\nI assume this paragraph is talking about the CelebA dataset, although it is never defined what \"natural RGB images\" are considered, beyond mentioning \"faces\". If this paragraph is about the CelebA experiment, I would suggest moving it before Table 1 is presented because it includes details about how those experiments are carried out.\n\n---\n\n**Conclusions**\n\n> ... plagued by the challenges of sparse measurements.\n\nI would consider rewording this sentence.", "broader_impact_concerns": "N/A", "additional_comments": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "xdzK479Xgw", "reviewer": "Reviewer_bPhg", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "**Strengths:**\n\n- The method is novel and introduces an interesting approach to using implicit neural representations for joint reconstruction.\n- Comprehensive ablation studies are conducted, thoroughly examining potential pitfalls such as overfitting, convergence issues, and computation time.\n- The paper is well-motivated, clearly written, and has a very clear and logical structure, making it easy to follow and understand.\n- Limitations of the method are clearly discussed.\n\n**Weaknesses:**\n- The benchmarks are limited and do not cover standard approaches, such as classical regularization methods, which are essential for a comprehensive evaluation (e.g. total generalized variation, wavelets regularization, patch-based tensor representations).\n- The authors use RGB images \"subjected to CT\" as a \"broader application\" example. However, this example is artificial and does not convincingly demonstrate the broader applicability of the method. For a proper evaluation of the broader application, the authors should demonstrate its performance in real-world scenarios such as MRI or optoacoustics.\n- Some claims such as for example: \"This computational overhead yields **substantial** gains in reconstruction quality and robustness across diverse settings\" are too strong according to the results tables and figures and should be toned down.", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "No", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "The authors propose to use implicit neural representations to improve reconstruction quality through joint reconstruction of multiple objects using implicit neural representations. They introduce a Bayesian framework that integrates latent variables to capture relationships between objects. The paper focuses on reconstructing sparse-view CT images and compares the method with other techniques, including Filtered Back Projection (FBP), iterative methods, some existing INR-based joint reconstruction techniques, a few-shot learning approach (MAML), and a federated learning approach (FedAvg). They demonstrate the results on several CT datasets, a synthetic dataset.", "claims_and_evidence_yn": "No", "claims_explanation": "", "audience_yn": "Yes", "audience_explanation": "", "requested_changes": "- Comprehensive benchmarks should include state-of-the-art regularization-based approaches such as total generalized variation, wavelet regularization, and patch-based tensor representations (e.g. https://onlinelibrary.wiley.com/doi/full/10.1002/mrm.27694).\n- Demonstration of broader impact on some real-world scenarios is necessary to validate the method's general applicability.", "broader_impact_concerns": "No concerns.", "additional_comments": "", "extra_scores": {}, "extra_text": {}}]

Author rebuttal:

Meta-review: The consensus is clearly that the paper should be accepted.
The only outstanding comment is  the narrow scope of the paper. The reviewers comments  do not imply that certifications should be awarded.
Decision: Accept (poster)