DiffusionBlend: Learning 3D Image Prior through Position-aware Diffusion Score Blending for 3D Computed Tomography Reconstruction
Authors: Bowen Song
Jason Hu
Zhaoxu Luo
Jeffrey A Fessler
Liyue Shen
Venue: NeurIPS 2024 poster (2024)
Keywords: diffusion models
CT reconstruction
inverse problems
Primary area: diffusion_based_models
TL;DR: We propose a novel algorithm that trains diffusion models on some 3D batches and blend the scores together for sparse-view CT reconstruction

Abstract: Diffusion models face significant challenges when employed for large-scale medical image reconstruction in real practice such as 3D Computed Tomography (CT).
Due to the demanding memory, time, and data requirements, it is difficult to train a diffusion model directly on the entire volume of high-dimensional data to obtain an efficient 3D diffusion prior. 
Existing works utilizing diffusion priors on single 2D image slice with hand-crafted cross-slice regularization would sacrifice the z-axis consistency, which results in severe artifacts along the z-axis. 
In this work, we propose a novel framework that enables learning the 3D image prior through position-aware 3D-patch diffusion score blending for reconstructing large-scale 3D medical images. To the best of our knowledge, we are the first to utilize a 3D-patch diffusion prior for 3D medical image reconstruction. 
Extensive experiments on sparse view and limited angle CT reconstruction
show that our DiffusionBlend method significantly outperforms previous methods
and achieves state-of-the-art performance on real-world CT reconstruction problems with high-dimensional 3D image (i.e., $256 \times 256 \times 500$). Our algorithm also comes with better or comparable computational efficiency than previous state-of-the-art methods. Code is available at https://github.com/efzero/DiffusionBlend.

DiffusionBlend: Learning 3D Image Prior through Position-aware Diffusion Score Blending for 3D Computed Tomography Reconstruction
Bowen Song
Jason Hu
Zhaoxu Luo
Jeffrey A. Fessler
Liyue Shen
Department of Electrical and Computer Engineering University of Michigan Ann Arbor, MI 48109
(bowenbw, jashu, luozhx, fessler, liyues @umi ch edu
Abstract
Diffusion models face significant challenges when employed for large-scale medi- cal image reconstruction in real practice such as 3D Computed Tomography (CT). Due to the demanding memory, time, and data requirements, it is difficult to train a diffusion model directly the entire volume of high-dimensional data obtain an efficient 3D diffusion prior. Existing works utilizing diffusion priors on single 2D image-slice with hand-crafted cross-slice regularization would sacrifice the z-axis consistency, which results in severe artifacts along the z-axis. In this work, we propose novel framework that enables learning the 3D image prior through position-awary 3D-patch diffusion score blending for reconstructing large-scale 3D medical images. To the best of knowledge, are the first utilize -a-patch diffusion prior for 3D medical image reconstruction. Extensive experiments on sparse view and limited angle CT reconstruction show that BiffusionBlend method significantly outperforms previous methods and achieves state-of-the-ee performance on real-world CT reconstruction problems with high- -igh-dimessional 3D image (i.c., 256 256 x 500). Our algorithm also comes with better or comparable computational efficiency than previous state-of-the-ar methods. Code available at: https::t///.o.oc..... /
E fuus onBlend.
Introduction
Diffusion models learn the prior of an underlying data distribution, which enables sampling from the distribution to generate new images [1-3]. By starting with clean image and gradually adding noise of different scales, diffusion sampler eventually obtains an image that is indistinguishable from pure noise. Let be the image sequence where 0 represents the clean image and /. = T' is pure noise. The score function of the image distribution, denoted as s(x;) = log p(xz), can be learned by a input and then approximates V log p(æ:). The
neural network parametrization. which takes reverse process then starts with pure noise and uses the learned score function to iteratively remove noise, ending with clean image sampled from the target distribution p(æ)).
Leveraging the learned score function as a prior, is efficient to solve the inverse problems based on diffusion priors. Previous works have proposed to use diffusion inverse solvers for deblurring, super-resolution, and medical image reconstruction such as in magnetic resonance imaging (MRI) and computed tomography (CT), and many other applications [4-16].
'Theee authors contributed equally to the work
38th Conference on Neural Information Processing Systems (NcurIPS 2024).

---

DOSIDifusionMBIR
DiftusionBlend(44 (Durs)
eleti (o4%)
Fv
stel
TV
alett 19
%
24
Figure 1: Overview of DiffusionBlend++ compared to previous 3D image reconstruction works. Previous work used a hand-crafted TV term to "regularize" adjacent slices, whereas the proposed approach uses learned diffusion score blending between groups of slices. Here i is the slice index, and is the reconstruction iteration.
Computed tomography (CT) reconstruction an important inverse problem that aims at reconstructing the volumetric image æ from the measurements y, which is acquired from the projections at different view angles [17]. To reduce the radiation dose delivered to the patient, sparse-view CT uses a smaller fraction of X-rays compared to the full-view CT [18]. Additionally, limited-angle CT is useful in cases where patients may have mobility issues and cannot use full-angle CT scans [19]. Although previous works have discussed and proposed diffusion- CT image reconstruction problem to demonstrate the proof-of-coneey [9. 10]. there is very limited work focusing on solving inverse problems for 3D images due to the practical difficulty in capturing 3D image prior. Learning efficient 2D image priors using diffusion models is already computationally expensive, which requires large-scale of training data, training time, and GPU memory. For example, previous works [2. 3] require training for several days to weeks on over a million training images in the ImageNet [20] and LSUN [21] datasets to generate high-quality 2D natural images of size 256 256. Hence, directly learning 3D diffusion prior on the entire CT volume would be practically infeasible or prohibitively expensive due the demanding requirements of training data and computational cost. In addition, real clinical CT data is usually limited and scarce and often has a resolution larger than 256 x 256 400, which makes directly training the data prior very challenging. The problem of tackling 3D image inverse problems, especially for medical imaging remains a challenging open research question.
methods for solving the 2D
few recent works [13-15] have discussed and proposed to solve 3D image reconstruction problems either through employing some hand-crafted regularization enforce consistency between 2D slices when reconstructing 3D volumetric images [13, 15], or through training several diffusion models for 2D images on each plane (axial, coronal, and sagittal), and performing reverse sampling with each model alternatively [14]. However, all of these works only learn the distribution of a single 2D slice via the diffusion model, while having not yet explored the dependency between slices that is required to better model the real 3D image prior.
To overcome these limitations, learning the distribution of 3D image patches (a batch of nearby 2D slices), and blends the scores of patches to model the entire 3D volume distribution for image reconstruction. Specifically, we firstly propose to train a diffusion model that models the joint distribution of 3D image patches (nearby 2D slices) in the axial plane conditioning on the slice thickness. Then, we introduce a random blending algorithm that approximates the score function of the entire 3D volume by using the trained 3D-patch score function. Moreover, we can either directly use the trained model predict the noise of single 2D slice by taking its corresponding 3D patch as input, or applying a random blending algorithm that firstly randomly partitions the volume into different 3D patches at each time step and then computes the score of each 3D patch during reverse sampling. Through either way, we can output the predicted noise of the entire 3D volume. In this way, our proposed method is able enforce cross-slice consistency without any hand-crafee regularizer. Our method has the advantage of being fully dataen and can enforce slice consistency without the TV regularizer as demonstrated in Fig. 1. Through exhaustive experiments of ultra-sparse-vie -parse-view and limited--ngee angle 3D CT reconstruction on different datasets, we validate that our proposed method achieves superior reconstruction results for 3D volumetric imaging, outperforming previous sffe-oofthh-ane (SOTA) methods. Furthermore, our method achieves better or comparable inference time than SOTA methods, and requires minimum hyperparameter tuning for different tasks and settings.
propose novel method, called DiffussioBBennd, that enables

---

In summary, our main contributions are as follows:
We propose DiffusionBlend((+)). a novel method for 3D medical image reconstruction through 3D diffusion priors. To the best of our knowledge, our method is the first diffusion- based method that learns the 3D-patch image prior incorporating the cross-slice dependency, as to enforce the consistency for the entire 3D volume without any external regularization.
Specifically, instead of independently training a diffusion model only on separated 2D slices, we propose novel method that first trains diffusion model on 3D image patches (a batch of nearby 2D slices) with positional encoding, and at inference time, employs a new approach of random partitioning and diffusion score blending to generate isotropically smooth 31) volume.
Extensive experiments validate our proposed method achieves state-of-the-art reconstruc- tion results for 3D volumetric imaging for the task of ultra-spare-vvew and limited-angle 3D CT reconstruction on different datasets, with improved inference time efficiency and minimal hyperparameter tuning.
2 Background and Related Work
Diff'usion models. Diffusion models consists of a forward process that gradually adds noise to a clean image, and reverse process that denoises the noisy images [1, 22]. The forward model is given by I2 = It-- P. z + Att where w E [N,0, 1) and B(t) is the noise schedule of the process. The distribution of æ: (0) is the data distribution and the distribution of aT)) is approximately standard Gaussian. When we set A + O, the forward model becomes dæ; = 3rx,d + Biddrr, which is a stochastic differential equation. The solution of this SDE given by
(L) Vt) log (x.)) dt + B(t)dw. 2
dr: =
Thus, by training neural network to learn the score function V.,, log (zx),, one can start with noise and run the reverse SDE to obtain samples from the data distribution.
Although diffusion models have achieved impressive success for image generation, a bottleneck of large-scale computational requirements including demanding training time, data, and memory prevents training diffusion model directly on nigh-dimensional high-resolution images. Many recent works have been studying how to improve the efficiency of diffusion models to extend them to large-scale data problem. For example, to reduce the computational burden, latent diffusion models [23] have been proposed, aiming to perform the diffusion process in a much smaller latent space, allowing for faster training and sampling. However, solving inverse problems with latent diffusion models still a challenging task and may have sub-par computational efficiency [24]. Very recently, various methods have been proposed to perform video generation using diffusion models, generally by leveraging attention mechanisms across the temporal dimension [25-28]. However, these methods only focus video synthesis. Utilizing these complicated priors for posterior sampling is still challenge because if these methods were applied to physical 3D volumes, continuity would only be maintained across slices in the XY plane and not the other two planes. Finally, work has been done to perform sampling faster [29-31], which is unrelated to the training process and network architecture. However, although these methods effectively promote the efficiency of training diffusion model, current works are not yet able to tackle the large-scale 3D image reconstruction problem real world settings.
3D CT reconstruction Computed tomography (CT) is a medical imaging technique that allows 3D object to be imaged by shooting X-rays through it [17]. The measurements consist of a set of 2D projection views obtained from setting up the source and detector at different angles around the object. By definition, y is the (known) set of projection views, A is the (in most cases assumed to be) linear forward model of the CT measurement system, and x is the unknown image. The CT reconstruction problem then consists of reconstructing given y. Traditional methods for solving this include regularization-bast methods that enforce a previously held belief on æ and likelihood based methods [17, 32-34].
Data-driven methods have shown tremendous success in signal and image processing in recent years [35-39]. In particular, for solving inverse problems, when large amounts of training data
3

---

PE:
Dendiing U-Nett
Demolsing U-Net
1-1
y
y
Sinogram
Denolsing step
Positional Encoder
Input ditfusion
Figure 2: Overview of slice blending process during reconstruction for DiffusionBlend++ At each iteration, partition the slices of the volume in a different way; slices of the same color inputted into the network independently. Positional encoding (PE) is also inputted to the network as information about the separation between the slices.
available, learned prior can be much stronger than the hand-crafted priors used in traditional methods [40, 41]. J. For past few years, many deep learning-based method have been proposed for solving the 3D CT reconstruction problem [42-45]. These methods train convolutional neural network, such as a U-Net [42], that maps the partial-view filtered backprojection (FBP) reconstructed image the ground truth image, that is, full-view CT reconstruction. However, these methods often generate blurry images and generalizes poorly for -istti-diitiibttion data [46].
3D CT reconstruction with diffusion models. Diffusion models serve very strong prior as they can generate entire images from pure noise. Most methods that use diffusion models solve inverse problems formulate the task conditional generation problem [47-49] or as posterior sampling problem (4-6, 50]. In the former case, the network requires the measurement y (or an appropriate resized transformation of y) during training time. Thus, at reconstruction time, that trained network can only be used for solving the specific inverse problem with poor generalizability. In contrast, for the posterior sampling framework, the network learns unconditional image prior for æ that can help solve various inverse problem related to æ without retraining. Although these diffusion-baat bassed methods have shown great performance for solving inverse problems for 2D images in different domains, there are seldom methods that are able to tackle inverse problems for 3D images because of the infeasible computational and data requirements aforementioned. Specifically, for 3D CT reconstruction, DiffusionMBIR [13] trains a diffusion model on the axial slices of volumes; at reconstruction time, it uses the total variation (T') regularizer with posterior sampling approach to encourage consistency between adjacent slices. Similarly. DDS [15] builds on this work by using accelerated methods of sampling and data consistency to greatly reduce the reconstruction time. However, although the TV regularizer has shown some success in maintaining smoothness across slices, it is data-driven method and does properly learn the 3D prior. TPDM [14] addresses this problem by training a separate prior on the coronal slices of volumes with conditional sampling approach, which serves as a data-driven method of maintaining slice consistency reconstruction time, but requires that all the volumes have the same cubic shape. In exchange, this method sacrifices the speed gains made by DDS, requiring alternating updates between the two separate priors, and is also twice as computationally expensive at training time. To overcome these limitations, we aim to propose a more flexible and robust approach that can learn the data prior properly for CT reconstruction, maintaining slice consistency while not sacrificing inference time.
Methods
Instead of modeling the 2D slices of the volume as independent data samples during training time, and then applying regularization between slices at reconstruction time, we propose incorporating information from neighboring slices at training time to enforce consistency between slices. More precisely, our first approach models the data distribution of 3D volume with H slices in the

---

dimension as follows:
p(z) II. H
p(=] | | -j j:i 11, 1. 1: j1)/2,
(2)
where j is a positive integer indicating the number of neighboring slices above and below the target slice that are being used as conditions to predict the target slice, and Z is a normalizing constant. To deal with boundary conditions where the third index may exceed the bounds of the original volume, we apply repetition padding above and below the main volume.
For training, we simply concatenate each of the conditioned slices with the target slice along the channel dimension to serve as an input to the neural network. Then we apply denoising score matching to predict the noise of the target slice as the loss function of the neural network:
(, B( (e [ |lea (xalis i j :i ox)
e(i, i]122
At reconstruction time, the score function of the entire volume decomposes as a sum of score functions of each of the slices:
v (())( R
V log loge( [ i - -j 1], xt :, ::: -))-
In this way, have rewritten the score of the 3D volume as sums of the scores of the 2D slices learned by the network. This means that we can now apply any algorithm that uses diffusion models to solve inverse problems solve the 3D CT reconstruction problem. Furthermore, this method of blending together information from different slices allows us to learn a prior for the entire volume that combines information from different slices. We call this method DiffusionBlend.
To learn an even better 3D image prior, instead of learning the conditional distribution of individual target slices, we can learn the joint distribution of several neighboring slices at once, which we call a 3D patch. Letting & be the number of slices in each patch, we can partition the volume into 3D patches and approximate the distribution of the volume
p(x)= I
| p(z|:,;, :, (i 1)k + ik)))/Z
Hík
(5)
where Z is a normalizing constant. Comparing this with (2), the main difference is instead of conditioning on neighboring slices, we are now incorporating the neighboring slices as joint distribution. This allows for much faster reconstruction, as k slices are updated simultaneously according to their score function. However, this method faces similar slice consistency issues as in [13], since certain pairs of adjacent slices (namely, pairs whose slice indices are congruent to 0 and 1 modulo are never updated simultaneously by the network.
To deal with this issue, we propose two additional changes. Firstly, instead of using the same partition (updating the same k slices) once for each iteration, we can use a different partition so that the previous border slices can be included in another partition. For example, we can randomly sample the end index of the first 3D patch for adjacency slices. Let m be uniformly sampled from 1, 2, ...,.. we can use the partition
S= (1,,...... H) =1, (I....,m) m] U (m + 1, m + k} U... U
1, H). (6)
instead of S = (1,2, the offset index number in the new partition. We can then compute the score on the new partition. More generally, we can choose an arbitrary partition of S into H/k sets, each containing k elements for each iteration, updating each slice in the small set simultaneously for that iteration.
(1 (
2 UH -+ H), where m
Secondly, to better capture information between nonadjacent slices, we apply relative positional encoding as an input to the network. More precisely, if 3D patch has slice thickness (the distance between two slices) of p. then we let be input of the positional encoding for that 3D patch. The positional encoding block consists of a sinusoidal encoding module and several dense connection modules, which has the same architecture as the timestamp embedding module of the same diffusion model. this manner, the network is able to learn how to incorporate information from nonadjacent slices and captures more global information about the entire volume. Recall that for 3D patches of adjacent slices, the border between patches may have inconsistencies. To address this, we can concatenate each border as new 3D patch, and then compute the score from it. If there are k slices in an adjacency-slice 3D patch, then the new 3D patch has the relative positional encoding of k. and also has size of k. For instance, if the previous partition is (4..))). the new partition
5

---

is (1.447),222,88))3...)))) Here we are forming new partition with jumping slices. In practice, since we need pretrained natural image checkpoint due scarcity of medical image data, we set & for facilitating fine tuning from natural image checkpoints.
We call the partitioning by 3D patch with adjacent slices Adjacency Partition, and the partitioning by 3D patch with jumping slices as Cross Partition. Letting r H/k be the number of 3D patches, with random partition, this method is stochastically averaging the different estimations of the V log p(æ) by different parititions. Specifically, the estimation of score by a single partition S1 U....US, S, is given by Σ V Vlogp(xl:: S.l). Ideally, we want to compute
SS| |S|_ E S= S.Suu..
Σ
Vlogp(al::;,S.))
Similar to [4, 13, 51], we can share the summation in (7) across different diffusion steps since the difference between two adjacent iterations z; and T441 is minimal.
In summary, we have shown how the score function of the entire volume can be written in terms of scores of the slices of the volume. Hence, similar to DiffusionBlend. this method can be coupled with any inverse problem solving algorithm. The scores of the slices can be approximated using neural network. Training this network consists of randomly selecting slices from a volume and concatenating them along the channel dimension to get the input to the network (along with the positional encoding of the slices), and then using denoising score matching as in (3) as the loss function; Section A.I provides theoretical justification for this procedure.
Sampling and reconstruction. With Eq. 7, each reconstruction step would require computing the score functions corresponding to each of the partitions of S, and then summing them to get the score function szz). We propose the variable sharing technique for this method, and only need to compute the score of one partition per time step. Hence, each iteration, we instead randomly choose one of the partitions of S and update the volume of intermediate samples by the score function. Finally, we use repetition padding if H is not a multiple of k. This method incorporates a similar slice blending strategy as DiffusionBlend, but allows for significant acceleration at reconstruction time k slices are updated at once. Furthermore, it allows the network to learn joint information between slices that farther apart without requiring the increase in computational cost associated with increasing k. We call this method DiffusionBlend++: The pseudocode of the algorithm can be found in Alg. 1.
In practice, we choose not to select from all possible partitions, but instead select from those where the indices each S, are not too far apart, as the joint information between slices that are very far apart is hard to capture. Table 12 summarizes the different 3D image prior models. The appendix provides more details about the partition selection scheme.
Krylov subspace methods. Following the work of [15], we apply Krylov subspace methods to enforce data consistency with the measurement. At each timestep t, by using Tweedie's formula [52], we compute é [x,), and then apply the conjugate gradient method
# = CG(A A, y, Lt, M):
(8)
where in practice, the CG operator involves running CG steps for the normal equation A'y Az. We combine this method with the DDIM sampling algorithm [29] to decrease reconstruction time. To summarize, we provide the algorithm for DiffusionBlend++ below. The Appendix provides the training algorithms for our proposed method as well the reconstruction algorithm for DiffusionBlend.
4 Experiments
Experimental setup. We used the public CT dataset from the AAPM 2016 CT challenge [53] that consists of 10 volumes. We rescaled the images in the XY-plane to have size 256 256 without altering the data in the Z-direction and used 9 of the volumes for training data and the tenth volume as test data. The training data consisted of approximately 5000 2D slices and the test volume had 500 slices. We also performed experiments on the LIDC-IDRI dataset [54]. For this dataset, we first applied data preprocessing by setting the entire background of the volumes to zero We rescaled the images in the XY-plane to have size 256 x 256, and, to compare with the TPDM method, only took

---

Algorithm DiffusionBlend++
Require: Forward model A, sinogram y. hyperparameter k; CG iteration numbers M Initialize æt ~ (O ((0,o for : do
Randomly select partition S = = U... U. (if mod k O, then use cross partition.
otherwise use random adjacency partitions)
Compute the relative positional encoding PE, For each i compute :,( P.₃)
Compute s log p(æ+) using (7)
Compute E₁ E E2 [t]] using Tweedie's ' formula
Set d'c = CG(A' A, y, Φt, .M)
Sample - using dt and via DDIM sampling
end for Return æ.
FBP
FBP-UNet
DDS
DDS
Ours
Ground Truth
Figure 3: Results of CT reconstruction with 4 views on AAPM dataset, axial view.
the volumes with least 256 slices the Z-direction, truncating the Z-direction to have exactly 256 slices. This resulted in 357 volumes which we used for training and one volume used for testing.
We performed experiments for sparse view CT (SVCT) and limited angle CT (LACT).. The detector size was set to 512 pixels for all cases. For SVCT, we ran experiments on 4, 6, and views. We also additional experiments
20, 40, 60, 80, and 100 views and report the quantitative results in the Appendix. For LACT, we used the full set views but only spaced around a 90 degree angle. In all cases, implementations the forward and back projectors can be found in [13].
For a fair comparison between DiffusionBlend and DiffusionBBend+++, we selected j = for DiffusionBlend and each S; to contain 3 elements for DiffusionBlend++. In this manner, both methods involve learning a prior that involves products of joint distributions on 3 slices. To train the score function for DiffusionBlend, consisted over 90000 slices, the network was able to properly learn this prior. We then fine tuned this network the much smaller AAPM dataset. For DiffusionBlend++, the input and output images both had 3 channels from stacking the slices, so we fine-tuned the existing checkpoint from [22]. All networks trained on PyTorch using the Adam optimizer with A40 GPUs. For reconstruction, we used 200 neural function evaluations (NFBs) for all the results. The appendix provides the full experiment hyperparameters. We observe that DiffusionBlend++ can reconstruct very high quality images that are free of artifacts as demonstrated in Fig.4 4 and Fig.3.
started from seratch using the LIDC dataset. Since this dataset
Comparison methods. We compared our proposed method with baseline methods for CT recon- struction and state of the art 3D diffusion model methods. We used the filtered back projection implementation found in [13]. For the other baseline, we used FBP-UNet [42] which supervised method that involves training network for each specific task mapping the FBP reconstruction to the

---

projections
40 projections
60 projections
80 projections
Ground Truth
Figure 4: Results of DiffussionBendd+ reconstruction with multiple views AAPM dataset, axial
view.
Sparse- View CT Reconstruction AAPM
Spanse-View
View CT Reconstruction LIDC
Method
views
views
views
Views
Views
Views
PSNR1
SSIM
PSNRT
SSIM1
PSNRT
SSIM
PSNRT
SSIM1
PSNR1
SSIM
PSNR1
SSIM1
FBP
14.66
0.359
13.65
0.293
11.94
0.222
14.79
0.217
14.11
0.191
13.18
0.169
FBP-UNet
26.00
0.849
24.15
0.782
23.37
0.761
28.58
0.848
26.48
0.781
25.19
0.731
DitlusionMBIR
26.30
0.863
24.99
0.827
23.66
0.789
32,67
0.922
31.18
0.901
29.02
0.863
TPDM
27.51
0.816
25.60
0,776
21.99
0.695
DDS 2D
32.89
0.946
31.40
0.934
28.77
0.906
30.82
0.897
29.38
0.867
27.54
0.826
DDS
33.19
0.945
31.94
0.942
29.22
0.916
31.65
0.915
30.12
0.888
27.20
8.8
DiffusionBiledd (Ours)
33.34
0.933
0.905
0.849
DitlusionBlend++ (Ours)
35.69
0,966
34.68
0.960
32,93
0.952
34.46
0.947
33.03
0.932
30.98
0.912
Table 1:
Comprehensive comparison of quantitative results on Sparse- View CT Reconstruction on Axial View for AAPM and LIDC datasets. Best results are in bold.
clean image. Since this is a 2D method, we learned mapping between 2D slices and then stacked the 2D slices to get the final 3D volume. such as SBTV, SIRT, and CGLS [55] to benchmark our algorithm against traditional methods. Results for these methods are reported in Appendix. For DiffusionMBIR [56), we fine-tuned the score function checkpoints on our data and used the same hyperparameters as the original work. We did the same for TPDM [14]; however, we ran TPDM only on the LIDC dataset because TPDM requires cubic volumes. Finally, were left unchanged (DDS), and another in which no TV regularizer between slices was enforced (DDS 2D). Both of these methods were with 200 NFEs. The appendix provides the experiment parameters.
also compared with classical CT reconstruction techniques
two variants of DDS [15]: one in which all the hyperparameters
Sparse-view CT. The results for different numbers of views and across different slices are shown in Tables 1, 11, and 3. DiffusionBlende4 exhibits much better performance over all the previous baseline methods (usually by a few dB) and outperforms DiffusionBlend. The second best method for each experiment is underlined and was, in most cases, DiffussinBBlndd The exceptions are when the second best method is DiffusionMBIRR but this method was run with 2000 NFEs and took about 20 hours to run compared to 1-2 hours for both of our methods. The two DDS methods required similar runtime as our methods but in all cases exhibited inferior reconstruction results. Furthermore, DDS 2D generally performed worse than DDS. Thus, DDS failed properly learn a 3D volume prior and still relied on the TV regularizer. Additionally, although TPDM should learn a 3D prior, the results were very poor compared to the other basclines. Our proposed method learned fully 3D prior and achieved the best results in the sagittal and coronal views.
Limited-angle CT. Table shows all results for limited angle CT reconstruction for both the AAPM and LIDC datasets. Our DiffusionBlend++ method obtains superior performance over all the baseline methods and DiffusionBlend obtains the second best results. Similar to the SVCT experiments, DiffusionMBII performed the best out of the baseline methods, but took approximately 40 hours to

---

Spanse-Vioe View
CT Reconstruction on AAPM
Sparse
Sparse-View CT Reconstruction on LIDE
Method
views
views
views
Views
PSNRT
SSIM
PSNRT
SSIM
PSNR1
SSIM
PSNRT
SSIM
PSNR1
SSIM
PSNR1
SSIM1
FBP
12.30
0.345
10.14
0.277
6.78
0.204
[4.88
0.234
14.30
0.207
13.43
0.187
FBP-UNet
26.13
0.860
24.14
0.798
23.47
0.779
28.56
0,848
26.52
0.783
25.29
0.732
DillusioMBBRR
26.64
0.869
25.08
0.834
23.71
0.789
52.79
0.922
31.30
0.900
28.98
0.862
TPDM
27.66
0.819
25.57
0.784
21.87
0.708
DDS 2D
33,22
0.949
31.69
0.937
29.39
0.909
30.98
0.894
29.40
0.862
27.54
0.819
DDS
33,43
0.945
52.18
0.947
29.86
0.924
31.80
0.915
30.13
0.889
27.26
0.818
DitlusionBlend (Ours)
35.09
0.952
0.945
33.73
0.934
31.16
0.907
27.93
0.855
DillusionBlend++ (Ours)
36.48
0.968
35.38
0.963
33.22
0.954
34,86
0.946
33,20
0.932
30.97
0.913
Table 2:
Comprehensive comparison of quantitative results on Sparse- View CT Reconstruction on Sagittal View for AAPM and LIDC datasets. Best results are in bold.
Sparse-View View CT Revonstruction AAPM
Spurse- View CT Reconstruction LIDe
Method
views
views
views
Views
Views
Views
PSNR1
SSIM
PSNR
SSIMF
PSNR1
SSIM
PSNRT
SSIM
PSNR1
SSIM
PSNR1
SSIM1
FBP
14.64
0.325
13.18
0,268
11.16
0.236
14.78
0.206
14.10
0.181
13.10
0.165
FBP-UNet
27.34
0.878
25.12
0.827
24.10
0.810
28.87
0.858
26.59
0.793
25.37
0.744
DillusioMMBR
29.86
0.908
28.12
0.875
25.68
0.843
33.29
0.922
31.69
0.903
29.21
0.868
TPDM
28.12
0.833
25.78
0.804
22.29
0.735
DDS 2D
33.64
0.950
32.33
0.939
30.25
0.916
31.60
0.898
29.99
0.871
28.03
0.830
DDS
33,97
0.934
32.95
0,930
30.89
0.932
32.51
0.920
30.83
0.898
27.61
0.828
DillusionBlend (Ours)
36.45
0.958
35.23
0.952
33.98
0.944
34.47
0.934
31,48
0,908
28.24
0.859
DillusionBBeed++ (Ours)
37.87
0.968
36.66
0,963
34,27
0.955
35,66
0.947
33.97
0.935
31,38
0.913
Table 3: Comprehensive comparison of quantitative results on Sparse- View CT Reconstruction on Coronal View for AAPM and LIDC datasets. Best results are in bold.
AAPM Dataset
LIDC
Method
Axial
Sagittal
Coronal
Asial
Sagittal
Coronal
PSNRT
SSIM
PSNR
SSIM
PSNR1
SSIM
PSNR
SSIM
PSNR1
SSIM
PSNR1
SSIM
FBP
16.36
0.643
16.36
0.524
13.62
0.531
18.79
0.672
19,84
0.675
20.01
0.676
PBP-UNet
27.38
0.910
27.81
0.918
28.44
0.930
29.42
0.885
29.50
0.884
29.54
0.887
DiflusionMBIR
25.98
0.872
27.14
0.877
27.74
0.903
30.52
0.901
30.57
0.916
30.68
0.907
TPDM
14.44
0.141
14.06
0.141
14.54
0.313
DDS 2D
28.05
0.916
27.99
0.916
28.82
0.922
27.92
0.843
27.8
0.835
27.96
0.842
DDS
28.20
0.918
28.17
0.926
29.03
0.934
28.12
0.865
28.06
0.869
28.13
0.879
DiffusionBlend (Ours)
35.38
0.971
35.85
0.972
37.62
0.972
30.43
0.917
31.24
0.920
31.02
0.924
DiflusionBlend++ (Ours)
0.975
0.976
37.45
0.976
34.33
0.957
34,48
0.957
34.64
0.956
Table 4: Comprehensive comparison of quantitative results on Limited- CT Reconstruction on All Views for AAPM and LIDC datasets. Best results are in bold.
run. FBP-UNet performed reasonably well, but is supervised method where the network must be retrained for each specific task. DDS is the most directly comparable to our method in runtime and methodology, but performed much worse quantitatively.
Inter-slice smoothness We demon- strate that DiffusionBlend++ learns the 3D prior internally, and achieves consistency and smoothness between 2D alial-plane slices without any external regularizations. In Table 5, we present the total variation (TV) value of the reconstructed images of different reconstruction algorithms on the test set of AAPM dataset, given by [D2 (2)]l1 where is the image, D is the total variation operator in z direction, and C. W. H are number of channels, width, and height. We find that both DiffusionBlend-+ and DDS have TV less than the ground truth image, which implies that the reconstructed images are smooth in the z direction. However, we observe that DDS over-smooths the images as demonstrated in Fig. 5, which is represented by a much lower TV value than the ground truth. On the other hand, Dif- fusionBlende has smoothness level close to the ground truth without sacrificing sharpness of images.
Algorithm
TV value Difference with gt
DDS 2D
0.0104
0.0044
DDS
0.0031
-0.0034
DiffusionBlend++ (Ours)
0.0043
-0.0022
Ground Truth
0.0065
Table 5: TV values of different reconstruction algorithms on the AAPM test set
CXW XH
9

---

Ground Truth Algorithm only with partitions of adjacency slices.
FBP
DiffusionPatch
DiffusionBlend+
DiffusionBlend++
Figure 5: Results of CT reconstruction with views AAPM dataset, coronal view. DiffusionPatch refers to Algorithm I with the same partition for every timestsep, and DiffusionBlend+ refers to
Effectiveness of adjacency-slice blending and cross-slice blending We demonstrate that both the adjacency-slice blending and the cross-slice blending module are instrumental to a better re- construction quality. Table demonstrates the effectiveness of adding blending modules to the reverse sampling. Given the pretrained diffusion prior over slice patches, we observe that adding the udjacency-sliee blending module improves the PSNR over a fixed partition by 1.17dB, and adding an additional cross-slice blending module further improves the PSNR by 1.63dB. Fig. 5 demon- strates that adding the cross-slice blending module removes artifacts and recovers sharper edges.
Ablation Studies We investigated the performance gain due to individual components. Details be found in Appendix A.3.
Adjacency Cross
PSNR
SSIM
34.85
0.954
36.02
0.965
36.48
0.968
Table 6: Effectiveness of Blending Mod-
Conclusion
ules, Sagittal view performance on AAPM
In this work, we proposed two methods of using score- based diffusion models to learn priors of three dimen- sional volumes and used them to perform CT recon-
struction. In both cases, we learn the distributions of multiple slices of a volume at once and blend the distributions together at inference time. Extensive experiments showed that our method substantially outperformed existing methods for 3D CT reconstruction both quantitatively and qualitatively in the sparse view and limited angle settings. In the future, more work could done on other 3D inverse problems and acceleration through latent diffusion models. Image reconstruction methods like those proposed in this paper have the potential to benefit society by reducing X-ray dose CT scans.
Acknowledgments and Disclosure of Funding
The authors acknowledge support from Michigan Institute for Computational Discovery and Engi- neering (MICDE) Catalyst Grant, and Michigan Institute for Data Science (MIDAS) PODS Grant.
References
Y. Song and S. Ermon. "Generative Modeling by Estimating Gradients of the Data Distribution". In: Advances Neural Information Processing Systems. Vol. 2019.
Y. Song. J. Sohl-Dickstein, D. Kingma, A. Kumar, Ermon, and B. Poole. "Score- Based Generative Modeling through Stochastic Differential Equations". In: 9th International Conference on Learning Representations, ICLR 2021, Virtual Event, Austria, May 3-7, 2021.
[3] Ho, A. Jain, and P. Abbeel. "Denoising Diffusion Probabilistic Models". In: 33 (2020), Ed. by Larochelle, M. Ranzato, R. Hadsell, M. Balcan, and H. Lin, pp. 6840-6851,
10

---

[4] H. Chung, J. Kim, M. I. Mccann, M. L. Klasky, a.d C. Ye. "Diffusion Posterior Sampling for General
Noisy Inverse Problems". In: The Eleventh International Conference on Learning Representations. 2023.
[5] B. Kawar, M. Flad, S. Ermon, and J. Song. Denoising Diffusioo Restoration Models. 2022.
[6] Y. Wang. J. Yu, and J. Zhang. Zero-Shot Image Restoration Using Denoising Diffuusion Null-Space Model.
2022.
[7] B. Kawar, G. Vaksman, and M. Elad. SNIPS: Solving Noisy Inverse Problems Stochastically. 2021.
[8] H. Chung, B. Sim, and J. C. Yc.
Faster:: Accelerating Conditional Diffusion Models
for Inverse Problems through Stochastic Contraction, 2022.
[9] H. Chung, B. Sim, D. Ryu, and J. C. Ye. Improving Diffusion Models for Inverse Problems using Manifold Constraints. 2022.
[10] Y. Song, L. Shen, L. Xing, and S. Ermon. "Solving Inverse Problems in Medical Imaging with -aooe-Based Generative Models". In: International Conference Learning Representations. 2022.
[11] A. Jalal, M. Arvinte, G. Daras, E. Price, A. G. Dimakis, and J. Tamir. "Robust compressed sensing mri with deep generative priors". In; Advances in Neural Information Processing Systems 34 (2021), pp. 14938-14954.
[12] W. Xia, H. W. Tseng, C. Niu, Cong, X. Zhang, S. Liu, R. Ning, S. Vedantham, and G. Wang. Parallel Sparse-view Cone-beam Breast CT. 2024.
Diffusson
[13] H. Chung, D. Ryu, M. McCann. M. L. Klasky, and C. Ye. Solving 3D Inverse Problems using Pre-truined 2D Diffusion Models. 2022.
[14] Lee, H. Chung, M. Park, J. Park, W.-S. Ryu, and C. Ye. Improving 3D Imaging with Pre- Pre-Trained Perpendicular 20 Diffusion Models. 2023.
[15] H. Chung, S. Lee, and C. Ye. Decomposed Diffusion Sampler for Accelerating Large-Scale Inverse Problems. 2024.
[16] W. Xia, W. Cong, and G. Wang. Patch-Based Denoising Diffussion Probubilistic Model for Sparse- Spurse-View
CT Reconstruction. 2022.
[17] L. A. Feldkamp, I.. C. Davis, J.d W. Kress. "Practical cone heam algorithm". In: J. Opt. Soc. Am. A
1.6 (Junc 1984), pp. 612-619.
[18] E. Y. Sidky, C.-M. Kao, and X. Pan. "Accurate image reconstruction from few-views and limited-angle data in divergent-beand CT In: Journal of X- Ray Science and Technology 14.2 (2006). pp. 119-139.
[19] T. M. Buzug. "Computed tomography". In: Springer handbook of medical technology. Springer, 2011, pp. 342.
[20]
O. Russakovsky et al. ImageNet Large Scale Visual Recognition Challenge. 2015.
[21] F. Yu, A. Seff, Y. Zhang. S. Song, T. Funkhouser, and J. Xiao. "LSUN: Construction of a Large-scale Image Dataset using Deep Learning with Humans in the Loop". In: arXiv preprint (2016).
[22] J. Ho, A. Jain, and P. Abbeel. "Denoising diffusion probabilistic models". In: Advances in Neural Information Processing Systems 33 (2020), pp. 6840-6851.
[23] R. Rombach, A. Blattmann, D. Lorenz, P. Esser, and Ommer. High-Resolution Image Synthesis with Latent Diffusion Models. 2022.
[24] B. Song, S. M. Kwon, Zhang, X. Hu, Q Qu, and Shen. Solving Inverse Problems with Latent Diffusion Models via Hard Data Consistency. 2023.
[25] A. Blattmann et Stable Video Diffasiom: Scaling Latent Video Diffission Models to Large Dutasets. 2023.
[26] Han, F. Kokkinos, and P. Torr. VFusion31D: Learning Scalable 3D Generative Models from Video Diffussion Models. 2024.
[27] Yu, K. Sohn, S. Kim, and J. Shin. Video Probabilistic Diffusion Models in Projected Latent Space. 2023.
[28] Y. Oshima, S. Taniguchi, M. Suzuki, and Y. Matsuo. SSM Meets Video Diffusion Models: Efficient Video Generation with Structured State Spaces. 2024.
[29] J. Song, C. Meng, and S. Ermon. "Denoising diffusion implicit models". In: arXiv preprint arXiv:2010.02502 (2020).
[30] T. Karras, M. Aittala, T. Aila, and S. Laine. Elucidating the Design Space of Diffusion- Rosed Generative Models. 2022.
[31] C. Lu. Y. Zhou, F. Bao, J. Chen, C. Li, and J. Zhu. "DPM-Solver: A Fast ODE Solver for Diffusson Probabilistic Model Sampling in Around 10 Steps". In: arXiv preprint TrXiv:2206.0092 (2022). [32] J.BB. Thibault, K. Sauer, C. Bouman, and J. Hsieh. "A three-dimensional statistical approach to improved image quality for multi-slice helical CT In: Med. Phys. 34.11 11 (Nov. 2007), pp. 4526-4544.
[33] J. Xu and B. M. W. Tsui. "Interior and sparse-view image reconstruction using mixed region and
voxel-basce ML-EM algorithm". In: IEEE Trans. Nuc. Sci. 59.5 (Oct. 2012), pp. 1997-2007.
11

---

[34] J. H. Cho and J. A. Fessler. "Regularization designs for uniform spatial resolution and noise properties in statistical image reconstruction for 3D X-ray CT In: IEEE Trans. Med. Imag. 34.2 (Feb. 2015).
pp. 678-689.
[35] J. Liu, Y. Sun, X. Xu, and U. S. Kamilov. Image Restoration using Total Variation Regularized Deep Image Prior. 2018.
[36] Z. Li, X. Xu, J. Hu, J. Fessler, and Y. Dewaraja. "Reducing SPECT acquisition time by predicting missing projections with single-scan sell-super coordinate- based learning". In: Journal of Nuclear Medicine 6uppupteeentt 1 (2023), P1014-P1014.
[37] Hu. B. T.-W. Lin, J. H. Vega, and R.-L. Tsiang. "Predictive Models of Driver Deceleration and Acceleration Responses to Lead Vehicle Cutting In and Out", In: Transportation Research Record 2677.5
(2023). pp. 92-102. DOI: 10. 1177/036 00//1161111 98 1111222222222
[38] X. Xu, W. Gan, S. V. V. N. Kothapalli, D. A. Yablonskiy, and U. S. Kamilov. CORRECT: A Deep Unfolding Framework for Motion Corrected Quantitative R2 Mapping. 2022.
[39] X. Xu, J. Liu, Y. Sun, B. Wohlberg, and U. S. Kamilov. "Boosting the Performance of Plug-and Play Priors via Denoiser Scaling". In: 54th Asilomar Conf. on Signals, Systems, and Computers. 2020, pp. 1305- 1312.
DOI: 10. 1119 IIEECONEEESSSS 394..99449 2020. 9443410.
[40] X. Xu, Y. Sun, J. Liu, B. Wohlberg, and U. S. Kamilov. "Provable Convergence of Plug-and-Play Priors With MMSE Denoiscrs". In: IEEE Signal Processing Letters 27 (2020), pp. 2020. 3006390.
DOI:
10. 1tw.119//100
[41] J. Liu, X. Xu, W. Gan, S. Shoushtari, and U. Kamilov, "Online Deep Equilibrium Learning for Reg- ularization by Denoising". In: Advances in Neural Information Processing Systems. Ed. by H. Oh,
A. Agarwal, D. Belgrave, and K. Cho. 2022.
[42] K. H. Jin, M. T. McCann, E. Froustey, and M. Unser. "Deep convolutional neural network for inverse problems in imaging". In: JEEE Transactions on Image Processing 26.9 (2017), pp. 4509-4522.
[43] A. Lahiri, G. Maliakal, M. L.. Klasky, J. A. Fessler, and S. Ravishankar. "Sparse-view cone heam CT reconstruction using data-consistent supervised and adversarial learning from scarce training data". In: IEEE Transactions on Computational Imaging 9 (2023), pp. 13-28.
[44] M. Sonogashira, M. Shonai, and M. liyama. #FfighResolutton Bathymetry by Seep-leaaming-Bae -Based Image Superresolution" In: PloS One 15.7 (2020), c0235487-60235487, DOI: 10. 1371 jjuunnll.. pone. 0235487.
[45] E. Whang. D. McAllister, A. Reddy, A. Kohli, and L. Waller. "SeidelNet: An Aberration-In Deep Learning Model for Spatially Varying Deblurring.. In: SPIE, Vol. 12438. 2023, 124380Y-124380Y-6 DOI: 10. 1117 / /12....22 2650416.
[46] V. Antun, F. Renna, C. Poon, B. Adcock, and A. C. Hansen. "On instabilities of deep learning in image reconstruction and the potential costs of Al", In: Proceedings of the National Academy Sciences 117.48 (2020), pp. 00088830099
[47] M. Delbracio and P. Milanfar. Inversion by Direct Iteration: An Alternative Denoising Diffusion for Image Restoration. 2024.
[48] G.-HL Liu, A. Vahdat, D.-A. Huang, E. A. Theodorou, W. Nie, and A. Anandkumar. FBB Image-to-Image Schrödinger Bridge. 2023.
[49] H. Chung, J. Kim, and C. Ye. Direct Diffusion Bridge using Data Consistency for Inverse Problems. 2023.
[50] G. Cardoso. Y. E. Idrissi. S. L. Corff, and E. Moulines. Monte Carlo guided Diffusion for Bayesian linear inverse problems. 2023.
[51] S. Lee, H. Chung, M. Park, J. Park, W.-S. Ryu, and C. Ye, "Improving 3D imaging with pre-trained perpendicular 2D diffusion models". In: Proceedings of the IEEE/CVF International Conference on Computer Vision 2023, pp. 10710-10720.
[52] B. Efron. "Tweedie's s formula and selection bias", In: Journal of the American Statistical Association 106.496 (2011). pp. 1602-1614.
[53] C. H. McCollough al. "Results of the 2016 Low Dose Grand Challenge". English (US). In: Medical physics 10 (Oct. 2017),
DO1: 10. 1002/000p/pp /mp.. 12345. [54] G. Armato, McLennan, L. Bidaut, M. F. McNitt-Gray and C. R. Meyer. "The lung image database
consortium (LIDC) and image database resource initiative (IDRI): A completed reference database of lung nodules on CT scans." In: Medical Physics 38 (2011), pp. 915-931.
[55] T. Goldstein and S. Osher. "The split Bregman method for L Ll-regularized problems". In: SIAM journal
on imaging sciences 2.2 (2009), pp. 323-343.
[56] H. Chung. B. Sim, D. Ryu, and J. C. Ye. "Improving Diffussin Models for Inverse Problems using Manifold Constraints". In: Advances in Neural Information Processing Systems. Vol. 35. 2022, pp. 25683- 25696.
[57]
Y. Han and J. C. Ye. "Framing U-Net via deep convolutional framelets: Application to sparse-view CT".
In: IEEE transactions on medical imaging 37.6 (2018), pp.
12

---

A Appendix supplemental material
A.1 Score matching derivations for DiflusionBlend++
We show how the score matching method described in [1] can be simplified in the case of assumptions such as the ones described Table 12.
Product of distributions. Suppose first that the distribution of interest can be expressed p(æ) ((x))"((=))
(x))//// /Z for density functions q and r, constant positive scalars and b, and scaling factor Z. Following [1], to learn the score function, we can minimize the loss function
2
where sy represents neural network. Denoting the score functions of p, q, and by 8p, and Sg. we have s(x) = (x) + (2:). Hence, if we instead use neural networks to learn 8p and 8q. we could minimize the loss function
x of
L
o1) b o1)
(10)
However, this loss function is computationally expensive to work with, as backpropagation through both networks is necessary. Thus, would be ideal to derive a simpler form of this loss function.
Toward these ends, for simplicity we define X a88,003,,00)
w- and Y (y), a+b of
b y- where all images have been vectorized. Now a+b of
X - 2 = | + || 2(X,Y) 20. 0.
(11)
Thus, rearranging the inequality and adding | + | / to both sides yields || + Y S 2| Y Returning to the original loss function, we have
L B Be By- + YII2.
(12)
By applying the inequality proven above, we get
y
Li S 21E Ea la s0
(13)
a 4 b
0
y x
+
(14)
2
For the special case of a = b = this inequality rewritten as
y x
Li S Ba- Byy
(15)
of
x di
+
(16)
Note that each of the two individual terms in the sum precisely represents the score matching equation for learning the score functions 8p and s,. Hence, train the networks 8y.0 and 8, by minimizing L1, we may instead minimize the upper bound of L1 by separately training these two networks.
In practice, may opt to the same network for 8..0 and s, but with additional input specifying which distribution between q and r to use. this case, at each training iteration, we randomly choose from one of the two distributions and perform backpropagation using this dis- tribution. More precisely, we redefine our network 8n 8n(x, ot, u) with e being either or 1. When so (x. oz, u) 8q.0 (2., ot) and when 1, s0 a,, oe, v) 8y,0 ot). Thus the loss bound becomes
y - F 12
L S B...
oz, Eve 0.,
(17)
o
13

---

Finally, this derivation easily extends to the more general case where the distribution of interest can be expressed as
p P. /Z.
(18)
In this case, the similarly defined score matching loss function L1 can be upper bounded by an expression similar to (17), but with v being randomly selected from & possible values.
In summary, we have shown that for the case of a decomposable distribution p(z), the score function of p(x) can be learned simply through the score function of the individual components p; (; . In the special case when each of the components have equal weight, it suffices to randomly choose one of the components and backpropagate through the score matching loss function according to that component.
Separable distributions. Next, we show how the score matching method is simplified for distri- butions of the form p(x)= = = S1 U...US, denotes an arbitrary partition of (1, ...,.... H). The score function of p(x) can be
[[== p( S,I) ZZ, where the same notation as Table 12 is used and
written
H
s (xx)
V Vlogp(a S.)
Σ 8, (=
(19)
where represents the score function of the slices of æ corresponding to S, Then (9) becomes
H
L Ben (an.off)
Σ
sa, (al S.)) y
(20)
of 2
Since each of the St's are disjoint, this can be broken up and rewritten
S:I) yl: S.
20 :. S
L = Σ
By- (0,1(
(21)
of
Thus, after replacing the outer sum with an expectation over this equivalent to randomly choosing one of the partitions S; and performing denoising score matching on only x[: :, Sy).
very similar derivation holds for the general case where the 3D volume arbitrary number of smaller volumes of any shape x æi UzzgU U. .. .Ux and p(z>) = I p(x;)/2 /Z. For this case, training consists of randomly selecting one of the partitions at each iteration and performing score matching on that partition. For example, when æ; æ/t.;, :, i). common to select 2D slices from the training volumes and learn a two dimensional diffusion model on those slices [13, 14].
be partitioned into an
Applying to BlelusionBlend++. When p(x) follows the distribution in DiffusionBlend++ we can combine the results of the previous two sections to show how to perform score matching. In the first part of this section, we showed how to perform score matching for distribution expressed as a product of "simpler" distributions by performing score matching on the individual distributions. DiffusionBlend++ follows this assumption where
(a(x:) [[ 'z..
(22)
Here, represents an index that can iterate through the ways of partitioning S = S₁ U... Sp. The input to the network specifying which of the simpler distributions is used is embedded as the relative position encoding for each of the partitions as described in Section 3. Finally, to learn the score function of p. (z)), we can use the loss function in (21).
A.2 Additional Algorithms
The reconstruction algorithm for DiffusionBlend is provided below.
The training algorithms for DiffusionBlend and DiffussonBBend++ are provided below.
14

---

Algorithm 2 DiffusionBlend
Require: A, M. <: > 0. j.
Initialize 2T ~ -N(00,
=ort= 1 do
For each i compute (z) ;, 2 1:1+
Compute 8 V log p(x;) using (4)
Compute E; E B/ mtg] using Tweedie's formula
Set d'; CG(A*A, A y. Øt).
Sample -t-1 using ey and via DDIM sampling
end for
Return
Algorithm 3 DiffussonBlend training
repeat
Select p(x)
Select f ~ Uniform(1, 7]
Set e NO, T)
Select i Uniform 1, H]
Take gradient descent step on Vxll(e((x,((,,, - - -j:::: j:i+j] σr) =((,;,
until converged
Return Dg
Algorithm 4 DifffusionBlend+ training
repeat
Select p(a)
Select t ~ Uniform [1, T
Set ~ (0,
Select a partition S = S, U...US,
Select i Uniform 1, r]
Take gradient descent step on Vel[(e((x((;; el: S.
until converged
Return Dg
15

---

A.3
Ablation studies
We run the following ablation studies to examine each of the individual components of our DiffusionBlende method. Firstly, we examine the performance gain of adding adjacency slicue blending (DiffusionBlend-) and adding cross-slice blending. Next, we examine the effect of including the positional encoding an input to the network. Then we look at the quantitative metrics of the reconstructed images when applying different numbers of NFEs for the comparison methods. Finally, we examine the effect of choosing different slices for each partition.
Adjacency Cross PSNR
SSIM
34.85
0.954
36.02
0.965
36.48
0.968
Table 7: Effectiveness of Blending Mod-
ules, Sagittal view performance AAPM
Effectiveness of adjacency-slice blending and cross-slice blending We demonstrate that both the adjaceney-slice blending and the cross-slice blending module are instrumental to a better re- construction quality. Table demonstrates the effectiveness of adding blending modules to the reverse sampling. Given the pretrained diffusion prior over slice patches, we observe that adding the adjacency-sii slice blending module improves the PSNR over a fixed partition by 1.17dB, and adding an additional cross-slice blending module further improves the PSNR by 1.63dB. Fig. 5 demonstrates that adding the cross-slice blending module removes artifacts and recovers sharper edges.
Robust performance with low NFEs. Since DDS and DiffusionBlend++ both the DDIM sampler for acceleration, we performed experiments with both of these methods using different NFEs. The left of Fig. 6 shows graphs of these two methods and Table 8 shows the quantitative results. DDS is very sensitive to the number of NFEs used and there is sharp dropoff in PSNR if too few or too many NFEs are used. On the other hand, DiffusionBlend++ performs the best for the highest number of NFEs due to the slice blending strategy while still obtaining superior results for 50 NFEs. Also, this method much more robust to varying NFEs, displaying only 1.4dB of variance in the shown results compared to 2.4dB of varaince for DDS. For fair comparisons, we use 200 NFEs for all the main experiments.
Frequency of applying slice jumps. To demonstrate
the use of jump slice partitions at reconstruction time, performed experiments varying the frequency of ap- plying these jump slices. For instance, for a frequency of 8, the reconstruction algorithm consisted of updating the volume using jump slices for iteration numbers that a multiple of 8 and updating using adjacent slices for all other iterations. The right of Fig. 6 shows graph of the results for different frequencies and the quantitative results are presented in Table 9. The best results are obtained when the frequency is 2, corresponding to alternating updates with adjacent slices and jump slices, and the PSNR decreases monotonically as the frequency increases. This indicates that the jump slices capture more nonlocal information across a volume and help to improve the image quality.
Table Axial PSNR for view SVCT re- con for different NFEs
Method
50
100
200
334
DDS
30.8 32,2 33.2 32.7
DiffusionBBledd+ 34.5 35.0 35.7 35.9
Table 9: Axial PSNR for 8 view SVCT recon for different slice jump frequencies
Frequency 2
4
8
12
16
32
PSNR 35.69 35.62
35.50
35.45
35.37
35.28
A.4 Additional Results
Classical Baselines and more Projection Angles We provide additional results with classical baselines (without deep learning) such as SIRT, SBTV, and CGLS for the LDCT dataset. Results show that our method outperforms the baselines significantly for every angle we evaluated on. DiffusionBBend++ starts to reconstruct images very close to the ground truth with 20 projections or more, but other baselines such as SIRT and CGLS still struggle to get satisfying reconstruction with
16

---

35.6
PSNR
35.5
DiffusionBlend++ (Ours)
35.3
DDS
NFEs
Skips steps for applying slice jump score
Figure 6: Quantitative results (axial view) of CT reconstruction with 8 views on AAPM dataset for different NFEs and slice blending methods.
Table 10: Wall times of various methods for 8 view 3D CT reconstruction
Method
NFEs
Wall time (min)
DiffusionMBIIR
2000
1400
TPDM
2000
1200
DDS
200
48
DiffusionBlend
200
70
DiffusionBlend++
200
32
60 views or more. Fig. shows the reconstruction performance on the coronal plane of different methods. We observe that DiffusionBlend++ (ours) has significant margin above baselines for every view. Note that DiffusionBlend++ still outperforms DDS2D significantly with >40 views, which demonstrates that our 3D prior is still very useful low-dose noise to the reconstruction, which showing our algorithm robust to noise by a minor decrease in reconstruction performance. Our method (DiffusionBlend++) is shown to outperforms all baselines at every angle as in Fig. 8.
with much more views. We also simulate
Error Bars We demonstrate the standard deviation of the results with sparse- view CT reconstruction on AAPM and LIDC dataset here to demonstrate that the result is statistically sigificant.
Fig. 9 shows the visual results for SVCT reconstruction with 8 views the LIDC dataset.
Fig. 10 shows the visual results for SVCT reconstruction with views on the LIDC dataset.
Fig. 11 shows the visual results for SVCT reconstruction with views on the LIDC dataset.
Fig. 12 shows the visual results for LACT reconstruction on the LIDC dataset.
DiffusionBlend++ with low-dose noise
DiffusionBlend++ without noise
42.5
Ground
25.0
36.0
CGLS
27.5
number
Figure 7: Left: Performance DiffusionBlend++ on more angles, Right: Reconstruction of Diffu sionBlend++ with low-dose noise
17

---

SBTV
SIRT
Ours
Ground Truth
Figure 8: Comparison of DiffusionBlend++ with classical methods
FBP-UNet
DiffusionMBIR
TPDM
DDS
DiffusionBlend
DiffusionBlend++
Ground truth
Figure 9: Results of 3D CT reconstruction with 8 views on LIDC dataset. Top row axial view, middle row is sagittal view, bottom row is coronal view.
FBP-UNet
DiffusionMBIR
TPDM
DDS
DiffusienBlend
DiffusionBlendt+
Ground truth
Figure 10: Results of 3D CT reconstruction with 6 views on LIDC dataset. Top row is axial view, middle is sagittal view, bottom row is coronal view.
18

---

Spanse View
Spans-VVeew
CT Reconstruction on AAPM
Sparse Beasee-View'
CT Reconstruction on LIDE
Method
views
views
PSNR1
SSIM
P
SSIM
PSNRT
SSIM
PSNR
SSIM
PSNR1
SSIM
PSNR1
SSIM1
FBP
2.64
0.16
2,88
0.26
2.91
0.20
2.57
0.17
2.78
0.21
2.84
0.16
FBP-UNet
3.46
0.30
3,34
0.28
3.04
0.31
3.42
3.23
3.14
DillusioMBBRR
1.93
0.08
1.48
0.13
1.28
0.11
1.89
0.09
1.56
0.12
1.31
0.14
TPDM
DDS 2D
1.76
0.07
2.04
0.10
2.73
0.11
1.85
0.09
2.13
0.13
267
0.18
DDS 2D
1.76
0.07
2.04
0.10
2.73
0.11
1.85
2.13
2.67
DDS 2D
1.68
0.06
1,96
0.09
2.65
0.11
1.84
2.10
0.12
2.64
0.18
DillusiomBlend (Ours)
1.67
0.06
1.78
0.08
1.98
0.09
1.70
0.07
2,03
0.11
2.54
0.16
DiffusionBlende+ (Ours)
Table 11: Standard Deviation of Performance Sparse-Viee View CT Reconstruction Sagittal View for AAPM and LIDC datasets. Best results are bold.
DiffusionMBIR
DiffuzionBlend++
Figure 11: Results of 3D CT reconstruction with 4 views LIDC dataset. Top row is axial view, middle row is sagittal view, bottom row is coronal view.
A.5 Experiment parameters
Since axial slices belonging the same volume that far apart have limited correlation, Diffu- sionBlend++ selects only partitions of for training where slices belonging to the same partition are fairly close to one another. Then the same range of possible partition schemes are used during reconstruction time. More precisely, we take the size of each S; to be and first repetition pad the volume so that the number of axial slices a multiple of 9. Then we consider the following partitions:
S (1,2,3) S2 6). S3 {7,8,9) Furthermore, for all integers A 1, S = Sx-3 ① 9 1)/3}, where + represents adding the same number to each element of the set. For example, S4 {10, 11, 12), Ss =113, 14, 15), S, {16, 17, 18).
S1 (1,4, 4. 7), S2 (2,5,8). S₃ (3.6, (3.6. Furthermore, for all integers k S = S + ( 1)/3/-
Table 12: 3D prior modeling methods
Method
Distribution Model
DiffusionMBIR [13]
Ih p( 1)/Z [o(l: : ))))) (II (a =))²) 12
TPDM [14]
DiffusionBlend
/1 p(=,,, d|| : 11, /1//
DiffusionBlend++
IT s
19

---

DiffusionMBIR
TPDM
DiffusionBlend++
Figure 12:
Results of limited angle 3D CT reconstruction on LIDC dataset. Top row is axial view, middle row is sagittal view, bottom row is coronal view.
A.6 Comparison experiment details
We used the same neural network architecture as the original paper [57]. Individual networks were trained for each of the 8 view, 6 view, 4 view, and LACT experiments for each of the datasets. Each of the networks were trained from scratch with batch size of 32 for 150 epochs.
FBP-UNet.
DiffusionMBIR.
We separately trained networks for the AAPM and LIDC datasets by fine-tuning the original checkpoint provided in [13] for 100 and 10 epochs, respectively. The batch size set to For reconstruction, we used the same set of hyperparameters for all of the experiments: À = 0.04, 10, and 0.16 for the sampling algorithm. 2000 NFEs were used for the diffusion process.
TPDM. We fine-tuned the axial and sagittal checkpoints provided in [14] on the LIDC dataset for 10 epochs. For reconstruction, we used 2000 NFEs and alternated between updating the volume using the axial checkpoint and sagittal checkpoint, with each checkpoint being used equally frequently. The DPS step size parameter was set to 0.5.
DDS. We separately trained networks for the AAPM and LIDC datasets by fine-tuning the original checkpoint provided in [15] for 100 and 10 epochs. respectively. We used 100 NFEs at reconstruction as this was observed give the best performance. The reconstruction parameters were set 1) = 0.85, 0.4, and 10. Five iterations of conjugate gradient descent were run diffusion step. For DDS 2D, the parameters were left unchanged with the exception of using p 0 to avoid enforcing the TV regularizer between slices.
SBTV. We implement this algorithm with variables splitting of 3D anisotropic TV regularization (Dz, Dx, and Dy). We first check number of iterations, note that the performance converges with around 30 iterations. We did grid search of hyperparameters on 9 validation images (not in the test set) for every projection angles.
SIRT. This algorithm iteratively updates the reconstruction based on the residual between projection of the reconstruction and the GT. It only has the number iterations as its hyper-parn parameter. We note that during inference, PSNR increases with more iterations, but saturates later. So set the total number iterations to be 1000, with an early stopping threshold of 1e-6 between two consecutive iterations.
This algorithm uses conjugate gradient for solving least square problems. In our case, we use CG(ATA A pzx x, AT "), p is set le-4 based on grid search for numerical stability. We tune the number of iterations validation set, and find that performance saturates at around 25 iterations.
CGLS.
20

---

A.7 Limitations
One limitation of our work is that we use noiseless simulated measurements for all our experiments. The robustness of our method to noise added to the measurements should be explored further. Likewise, future work should evaluate the accuracy of our method when applied to real measurement data, which will contain measurement noise and mismatches between the true system model and used forward model. Another limitation of our work lack of other types of 3D image reconstruction applications shown. Although the proposed method is unsupervised and the reconstruction algorithm can readily be applied other 3D linear inverse problems, future work should explore other applications of DiffusionBBend.
21

---

NeurIPS Paper Checklist
1. Claims
Question: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?
Answer: [Yes]
Justification: The abstract and introduction contain claims that are expounded upon in the remainder of the paper.
Guidelines:
The answer NA means that the abstract and introduction do not include the claims made the paper.
The abstract and/or introduction should clearly state the claims made, including the contributions made in the paper and important assumptions and limitations. A No or
NA answer to this question will not be perceived well by the reviewers.
The claims made should match theoretical and experimental results, and reflect how much the results can be expected to generalize to other settings.
is fine to include aspirational goals as motivation as long as is clear that these goals not attained by the paper.
2. Limitations
Question: Does the paper discuss the limitations the work performed by the authors?
Answer: [Yes]
Justification: Section A.7 outlines the limitations of the proposed method.
Guidelines:
The answer NA means that the paper has no limitation while the answer No means that the paper has limitations, but those are not discussed in the paper.
The authors are encouraged to create a separate "Limitations" section their paper.
The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g., independence assumptions, noiseless settings, model well-specification, asymptotic approximations only holding locally). The authors should reflect how these assumptions might be violated in practice and what the implications would be.
The authors should reflect on the scope of the claims made, e.g., if the approach was only tested on few datasets or with few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.
The authors should reflect on the factors that influence the performance of the approach. For example, facial recognition algorithm may perform poorly when image resolution is low images are taken in low lighting. Or speech-to-text system might not be used reliably to provide closed captions for online lectures because it fails to handle technical jargon.
The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size.
If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and fairness.
While the authors might fear that complete honesty about limitations might used by reviewers as grounds for rejection, worse outcome might be that reviewers discover limitations that aren't 't acknowledged judgment and recognize that individual actions in favor of transparency play impor- tant role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed to not penalize honesty concerning limitations.
the paper. The authors should use their best
3. Theory Assumptions and Proofs
Question: For each theoretical result, does the paper provide the full set of assumptions and complete (and correct) proof?
Answer: [Yes]
22

---

The paper does not contain any theorems, but some theoretical foundations for the algorithms used are provided in various appendices.
Justification:
Guidelines:
The answer NA means that the paper does not include theoretical results.
All the theorems, formulas, and proofs in the paper should be numbered and cross- referenced.
All assumptions should be clearly stated or referenced the statement of any theorems.
The proofs can either appear in the main paper or the supplemental material, but if they appear in the supplemental material, the authors are encouraged to provide short proof sketch provide intuition.
Inversely, any informal proof provided in the core of the paper should be complemented by formal proofs provided in appendix or supplemental material.
Theorems and Lemmas that the proof relies upon should be properly referenced.
4. Experimental Result Reproducibility
Does the paper fully disclose all the information needed to reproduce the main ex- perimental results of the paper to the extent that it affects the main claims and/or conclusions of the paper (regardless of whether the code and data are provided or not)?
Question:
Answer: [Yes]
Justification: The pseudocode for the algorithms as well as hyperparameter selection and datasets used are completely outlined in the main body and appendix.
Guidelines:
The answer NA means that the paper does not include experiments.
If the paper includes experiments, a No answer to this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.
If the contribution is dataset and/or model, the authors should describe the steps taken make their results reproducible or verifiable.
Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is novel architecture, describing the architecture fully might suffice, or if the contribution a specific model and empirical evaluation, it may be necessary either make it possible for others to replicate the model with the same dataset, or provide access to the model. In general. releasing code and data is often one good way accomplish this, but reproducibility can also be provided via detailed instructions for how to replicate the results, access to a hosted model (e.g., in the case of large language model), releasing of a model checkpoint, or other means that appropriate to the research performed.
While NeurlPS does not require releasing code, the conference does require all submis- sions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example
(a) the contribution is primarily a new algorithm, the paper should make clear how reproduce that algorithm. (b) If the contribution is primarily new model architecture, the paper should describe the architecture clearly and fully.
(c) the contribution is a new model (e.g., a large language model), then there should either be way to access this model for reproducing the results or way to reproduce the model (e.g., with an open-source dataset instructions for how to construct the dataset).
(d) We recognize that reproducibility may be tricky in some cases, in which case authors are welcome deseribe the particular way they provide for reproducibility. In the case of closed-source models, it may be that access to the model is limited in some way (e.g., to registered users), but it should possible for other researchers to have some path to reproducing or verifying the results.
5. Open access to data and code
23

---

Does the paper provide open access to the data and code, with sufficient instruc- tions faithfully reproduce the main experimental results, as described in supplemental material?
Question:
Answer: [Yes]
Justification: have released the code to a public Github repository linked the abstract and will be working over the next weeks to fully update it.
Guidelines:
The answer NA means that paper does not include experiments requiring code.
Please see the NeurlPS code and data submission guidelines (https: /ts:://nisss cc/
public /wwp///guss///eeeee Cooeeeuoeeiii soniissiinn ssionPolicy for more details.
While we encourage the release of code and data, we understand that this might be possible, so "No" is an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (e.g., for new open-source benchmark).
The instructions should contain the exact command and environment needed to run to reproduce the results. See the NeurIPS code and data submission guidelines (ht (https: tps:: //nips cc/ public/ /////uudisss Code as onn oooicyy icy) for more details.
The authors should provide instructions on data access and preparation, including how to access the raw data, preprocessed data, intermediate data, and generated data, etc.
The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. only subset of experiments are reproducible, they should state which ones omitted from the script and why.
At submission time, to preserve anonymity, the authors should release anonymized versions (if applicable).
Providing as much information as possible in supplemental material (appended to the paper) recommended, but including URLs to data and code permitted.
6. Experimental Setting/Details
Question: Does the paper specify all the training and test details (e.g., data splits, hyper- parameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?
Answer: [Yes]
Justification: All details of the experiments have been provided.
Guidelines:
The answer NA means that the paper does not include experiments.
The experimental setting should be presented in the core the paper to level of detail that is necessary to appreciate the results and make sense of them.
The full details can be provided either with the code, in appendix, as supplemental material.
7. Experiment Statistical Significance
Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?
Question:
Answer: [Yes]
We report the standard deviation of the main experimental results. Since results is the average of about hundreds of test samples, given the standard deviation the result implies statistical significancy. Uncertainty metrics that were run. Since we use large-scale 3D data for the experiments, it would be very time consuming to run each experiment many times.
reported for some other the experiments
Guidelines:
The answer NA means that the paper does not include experiments.
The authors should answer "Yes" if the results are accompanied by emor bars, confi- dence intervals, or statistical significance tests, least for the experiments that support the main claims of the paper.
24

---

The factors of variability that the error bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall run with given experimental conditions).
The method for calculating the error bars should be explained (closed form formula, call to a library function, bootstrap, etc.)
The assumptions made should be given (e.g., Normally distributed errors).
It should be clear whether the error bar is the standard deviation or the standard error of the mean.
is OK to report 1-sigma error bars, but one should state it. The authors should preferably report a 2-sigma error bar than state that they have 96% CI, if the hypothesis of Normality of errors is not verified.
For asymmetric distributions, the authors should be careful not to show in tables or figures symmetric error bars that would yield results that are out range (e.g. negative еπτог rates).
If error bars are reported in tables or plots, The authors should explain in the text how they were calculated and reference the corresponding figures or tables text.
8. Experiments Compute Resources
For each experiment, does the paper provide sufficient information on the com- puter resources (type of compute workers, memory, time of execution) needed to reproduce the experiments?
Question:
Answer: [Yes]
Justification: The compute resources and runtime for the experiments are specified in the paper.
Guidelines:
The answer NA means that the paper does not include experiments.
The paper should indicate the type of compute workers CPU GPU, internal cluster, or cloud provider, including relevant memory and storage.
The paper should provide the amount of compute required for each of the individual experimental runs well estimate the total compute.
The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g., preliminary or failed experiments that didn't make into the paper).
9. Code Of Ethics
Question: Does the research conducted in the paper conform, in every respect, with the NeurIPS Code of Ethics https: /nwuripss.p..
Answer: [Yes]
Justification: We have read the Code of Ethics and checked that the research conducted in the paper conforms to it.
Guidelines:
The answer NA means that the authors have reviewed the NeurIPS Code of Ethics.
If the authors answer No, they should explain the special circumstances that require deviation from the Code of Ethics.
The authors should make to preserve anonymity (e.g., - if there is special consid- eration due to laws or regulations in their jurisdiction).
10. Broader Impacts
Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?
Answer: [Yes]
Justification:
We have mentioned the broader impacts of the work in the conclusion.
Guidelines:
The answer NA means that there is no societal impact of the work performed.
25

---

If the authors answer NA or No, they should explain why their work has no societal impact or why the paper does not address societal impact.
Examples of negative societal impacts include potential malicious or unintended uses (e.g., disinformation, generating fake profiles, surveillance), faimess considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.
The conference expects that many papers will be foundational research and tied to particular applications, let alone deployments. However, if there is direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that an improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, is not needed to point out that a generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster.
The authors should consider possible harms that could arise when the technology is being used intended and functioning correctly, harms that could arise when the technology is being used as intended but gives incorrect results, and harms following from (intentional unintentional) misuse of the technology.
If there are negative societal impacts, the authors could also discuss possible mitigation strategies (e.g.. gated release of models, providing defenses in addition attacks, mechanisms for monitoring misuse, mechanisms to monitor how a system learns from feedback over time, improving the efficiency and accessibility of ML).
11. Safeguards
Does the paper describe safeguards that have been put in place for responsible release of data models that have high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?
Question:
Answer: [Yes]
The models used in the paper can generate images, but the datasets used have been checked be safe. Since the models can only generate images similar to the datasets on which they have been trained, the images that can be generated should also conform to this safety.
Justification:
Guidelines:
The answer NA means that the paper poses no such risks.
Released models that have high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere usage guidelines or restrictions to access the model or implementing safety filters.
Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.
We recognize that providing effective safeguards is challenging, and many papers do not require this, but encourage authors to take this into account and make best faith effort.
Licenses for existing assets
Question: Are the creators or original owners of assets (e.g., code, data, models), used in the paper, properly credited and are the license and terms of use explicitly mentioned and properly respected?
Answer: [NA]
Justification: Data used in this paper is from public domain.
Guidelines:
The answer NA means that the paper does not use existing assets.
The authors should cite the original paper that produced the code package dataset.
The authors should state which version of the asset used and, possible, include URL.
The name of the license (c.g., CC-BY 4.0) should be included for each asset.
26

---

For scraped data from a particular source (e.g., website). the copyright and terms of service of that souree should be provided.
assets are released, the license, copyright information, and terms of use the package should be provided. For popular datasets, papers thoode com/ dat ssets has curated licenses for some datasets. Their licensing guide can help determine the license of dataset.
For existing datasets that are re-packaged, both the original license and the license of the derived asset (if it has changed) should be provided.
If this information is not available online, the authors are encouraged to reach out to the asset's s creators.
13. New Assets
Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?
Answer: [Yes]
Justification:
We have cited all works and datasets that this paper uses.
Guidelines:
The answer NA means that the paper does not release new assets.
Researchers should communicate the details of the dataset/code/mmodl as part of their submissions via structured templates. This includes details about training, license, limitations, etc.
The paper should discuss whether and how consent obtained from people whose asset is used.
At submission time, remember to anonymize your assets (if applicable). You can either create an anonymized URL or include an anonymized zip file.
14. Crowdsourcing and Research with Human Subjects
Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given participants and screenshots, if applicable, as well as details about compensation (if any)?
Answer: [NA]
Justification:
The paper does not involve crowdsourcing or research with human subjects.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.
Including this information in the supplemental material is fine, but if the main contribu- tion of the paper involves human subjects, then as much detail as possible should be included in the main paper.
According to the NeurlPS Code of Ethics, workers involved in data collection, curation, or other labor should be paid at least the minimum wage in the country of the data collector.
15. Institutional Review Board (IRB) Approvals or Equivalent for Research with Human Subjects
Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or an equivalent approval/review based on the requirements of your country or institution) were obtained?
Answer: [NA]
Justification:
The paper does not involve crowdsoureing or research with human subjects.
Guidelines:
The answer NA means that the paper does not involve crowdsoureing nor research with human subjects.
27

---

Depending on the country in which research is conducted, IRB approval (or equivalent) may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.
We recognize that the procedures for this may vary significantly between institutions and locations, and we expect authors to adhere to the NeurIPS Code of Ethics and the guidelines for their institution.
For initial submissions, do not include any information that would break anonymity (if applicable), such as the institution conducting the review.
28


Reviews:
[{"review_id": "vnQTpKP2zA", "reviewer": "Reviewer_YiqS", "summary": "This paper proposes a novel method for learning 3D diffusion priors for CT reconstruction, which does not require large-scale data or computational resources. It presents two approaches: DiffusionBlend and DiffusionBlend++. The former learns a specific frame given adjacent slices, while the latter learns a 3D patch. Experimental results demonstrate that both methods are efficient and outperform previous works.", "questions": "I don't have questions about this paper.", "limitations": "The authors have discussed the limitations.", "rating": 5, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "1. This method enables 3D reconstruction without the need for large-scale data and resources.\n2. It achieves excellent inter-slice consistency without relying on external regularizations.\n3. Additionally, this method outperforms existing baselines.", "weaknesses": "The only concern is that this method may be too simplistic to be extended to a broader field. For example, I believe that inter-slice smoothness cannot be guaranteed if this method is applied to videos. However, given that this method is proposed for CT reconstruction, I don't think this issue warrants the rejection of the paper.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thank you for providing the valuable feedback. Below, we address the concern:\n\n> **Q:** *Extending the method to a broader field*\n\nA: Our method is flexible and does not assume any specific data modality and forward models, so it should work for other modalities such as natural 3D images or videos, and for other inverse problems as well. Nevertheless, the results on other modalities and tasks are beyond the scope of this work, but we tested unconditional generation of 3D CT images which show good generation quality and inter-slice smoothness. We will include this experiment in the camera-ready version as reference. Thus, we think that the inter-slice smoothness of the learned prior from our method could be potential to work for videos as well to obtain a inter-frame coherence, which can be a very interesting future works. We will add this discussion in the camera-ready version as well.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "mGFZPQzegR", "reviewer": "Reviewer_Vnw7", "summary": "1. The method proposed a method that learns the 3D-patch image prior incorporating the cross-slice dependency.\n2. The method achieves state-of-the-art reconstruction results for 3D volumetric imaging for the task of ultra-sparse-view and limited-angle\n3D CT reconstruction on  \"AAPM 2016 CT challenge\" dataset and \"LIDC-IDRI\" dataset.", "questions": "1. If the \"conditioned slices\" in equation (2) increase(\"conditioned slices\" is: x[:,:,i-j:i-1] and x[:,:,i+1:i+j]), will the numerical performance better? If it is better, how does the performance gain change as the number of adjacent slices increases?\n\n2. The paper mentions using a different partition so that the previous border slices can be included in another partition. However, if the method creates a new partition and then uses this new partition to create 3D patches, the adjacent slices of different 3D patches still cannot be updated simultaneously when the algorithm attempts to update the slices based on the scores calculated in each 3D patch. So, my question is, if the method uses the joint distribution modelling method (Eq. (5)), how can it avoid the situation I mentioned?\n\n3.  For the experimental part,  The reconstruction mentioned here is the reconstruction from the simulated projection to the 3D CT volume or the reconstruction  from  the real CT projection(real measurement) to the 3D CT volume? If it is a simulated projection, is simulated noise added to the simulated projection? (Note: The noise I am referring to here is not the noise added in the diffusion model to train the neural network, but the noise added to simulate the real projection, which is generally a combination of Poisson noise and Gaussian noise.)", "limitations": "The artifacts generated by the diffusion model may lead to misjudgment in the doctor's diagnosis. Please provide methods to avoid artifact, such as increasing the angle or the amount of training data?", "rating": 6, "confidence": 4, "soundness": 1, "presentation": 3, "contribution": 3, "strengths": "1. The probabilistic modeling of the 3D volume taking into account neighboring slices is novel.\n2. The paper describes the author's motivation and implementation plan very well.", "weaknesses": "1. Sparse angle CT imaging is typically defined as using fewer than 100 angles. The paper should evaluate the method's performance at various angles (e.g., 20, 40, 60, 80, and 100 angles). Demonstrating that this method outperforms the comparison method in PSNR/SSIM at fewer than 10 angles does not necessarily indicate better performance at other sparse angles (e.g., 20, 40, 60, 80, and 100 angles).\n\n2. The comparison should include classic CT reconstruction methods, not just FBP, as FBP lacks prior information and performs poorly among traditional methods. Suggested comparison methods include:\n2.1 SIRT (Simultaneous Iterative Reconstruction Technique) algorithm\n2.2 Conjugate Gradient Least Squares (CGLS) algorithm\n2.3 Split-Bregman (SB) Total Variation: Goldstein, T. and Osher, S., 2009. The split Bregman method for L1-regularized problems. SIAM journal on imaging sciences, 2(2)\nBecause the traditional iterative method also involves the adjustment of hyperparameters, for a fair comparison, please first use grid search to search the hyperparameters of the traditional method to obtain the best performance of the traditional method, and then compare the performance of the traditional method with the performance of this paper. Also, please report the extent of the grid search. Only in this way can it can be fully evaluated in the numerous algorithms for CT image reconstruction.\n\n3. Figure 3 shows a large number of structures that do not exist in CT images, which will interfere with the doctor's diagnosis. Please explain the reasons for this phenomenon and how to avoid it. (For example, increase the number of angles, which is why I am very concerned about increasing the angles). If increasing the angles can avoid the artifacts that come with the generative model on a large scale, the potential for medical applications will increase.\n\n4. As the abstract said, the method's purpose is to decrease the cost of memory, time. In order to prove that this method has achieved the original intention of it, please show the time consuming and memory consuming during training phase and testing phase. And compare the memory and time consuming between different methods.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thank you for providing the valuable feedback. Below, we address the concern:\nFirst, we address concerns regarding to additional experiments:\n\n> **Q:** *Evaluate method's performance at various angles.*\n\nA: \n- We provide results on [20, 40, 60, 80, 100] views for our method (DiffusionBlend++) as well as baselines with those angles (including the SB(Split-Bregman)TV, CGLS, and SIRT baselines) for the LDCT dataset in the rebuttal pdf as well in the table below. Results show that our method ***outperforms the baselines significantly*** for ***every*** angle we evaluated on. From Figure 1,2,3 in the rebuttal pdf DiffusionBlend++ starts to reconstruct images very close to the ground truth with 20 projections or more, but other baselines such as SIRT and CGLS still struggle to get a satisfying reconstruction with 60 views or more. The following table shows the reconstruction performance on the coronal plane of different methods. We observe that DiffusionBlend++ (ours) has a significant margin above baselines for every view. Note that DiffusionBlend++ still outperforms DDS2D significantly with >40 views, which demonstrates that our 3D prior is still very useful even with much more views.\n- The motivation of this paper is to exploit the 3D data-driven prior (instead of external 3D regularization or learning another 2D prior on another plane) for medical image reconstruction. The hypothesis is that with the aid of the 3D prior, we can have better reconstruction quality comparing to only using 2D priors. Many previous works leverage 2D diffusion priors for sparse-view reconstructions ( > 10 views), but only a few works exploit reconstructions with fewer than 10 views. By this consideration, we do not provide results for more than 10 projection in the main paper. Nevertheless, we will add additional results with more projection angles and more traditional baselines in the camera-ready version.  \n``Table.1 Coronal plane results on LDCT test set``\n| Method           |4 views| 6 views| 8 views | 20 views | 40 views | 60 views | 80 views | 100 views |\n|------------------|----------|----------|----------|----------|----------|----------|-----------|-----------|\n| FBP |    11.16/0.236   |    13.18/0.268    |    14.64/0.325   |    20.22/0.594    |     26.09/0.792    |     29.97/0.886    |     32.89/0.933    |     35.16/0.959    |\n| SIRT |    22.12/0.823    |    23.54/0.841    |    24.50/0.851    |    27.72/0.882    |     30.82/0.918    |     33.34/0.945    |     35.56/0.962    |     37.66/0.974    |\n| CGLS |    22.11/0.823    |    23.49/0.841    |    24.44/0.850   |    27.67/0.882    |     30.66/0.918    |     32.96/0.944    |     34.82/0.960    |     36.36/0.971    |\n| SBTV |    23.33/0.849    |    25.83/0.881    |    27.85/0.896    |    31.51/0.938   |     34.83/0.966    |     36.74/0.963    |     37.59/0.973    |     38.97/0.979    |\n| DDS2D |    30.25/0.916    |    32.33/0.939    |    33.64/0.950   |    35.34/0.961    |     38.26/0.977    |     40.07/0.984    |     41.19/0.987   |     41.88/0.989    |\n| DDS |    30.89/0.932   |    32.95/0.879    |    33.97/0.935    |    35.81/0.967    |     37.59/0.975    |     38.26/0.978    |     39.07/0.980    |     39.54/0.983    |\n| DiffusionBlend++ |    ***34.27/0.955***    |    ***36.66/0.963***   |    ***37.87/0.968***    |   ***40.56/0.980*** |  ***42.53/0.987***    |     ***43.66/0.990***    |     ***44.42/0.992***    |     ***44.95/0.993***    |\n\n\n\n\n\n> **Q:** *Comparison with classic CT reconstruction methods, not just FBP*\n\nA: Table.1 and Figure.1,2,3 in the rebuttal pdf show more results on additional classic baselines. Note that DiffusionBlend++ ***does not need to tune any hyperparameter*** when the number of projections changes, but this is a requirement for some traditional methods. \n- SBTV: We implement this algorithm with variables splitting of 3D anisotropic TV regularization (Dz, Dx, and Dy). We first check number of iterations, note that the performance converges with around 30 iterations. We did a grid search of hyperparameters on 9 validation images (not in the test set) for every projection angles. For example: the searching for 20 projections is conducted as below:\n\n| $\\lambda$       |  $\\lambda / \\rho = 0.02$ | $0.04$ | $0.08$ | $0.16$ |\n|--------|------|------|------|------|\n| 1.25   |   27.88   | 28.34   |  28.30    |  27.88    |\n| 2.5    |  28.16    |  ***28.36***    |  28.07    |  27.44    |\n| 5      |  28.10    | 28.06     |  27.57    |   26.73   |\n| 10     |   27.98   |  27.57    | 26.77     |  26.24    |\n\n- SIRT: This algorithm iteratively updates the reconstruction based on the residual between projection of the reconstruction and the GT. It only has the number of iterations as its hyper-parameter. We note that during inference, PSNR increases with more iterations, but saturates later. So we set the total number iterations to be 1000, with an early stopping threshold of 1e-6 between two consecutive iterations.  \n\n- CGLS: This algorithm uses conjugate gradient for solving least square problems. In our case, we use $CG(A^TA + \\rho x^Tx, A^Ty)$, $\\rho$ is set to be 1e-4 based on grid search for numerical stability. We tune the number of iterations on validation set, and find that performance saturates at around 25 iterations.\n\n> **Q:** *Whether the projections have noise?*\n\nA: Following approaches in [1,2,3,4,5], we do not add noise in our projections since the noise is very small for normal dose. Nevertheless, we also run additional experiments with 8-view reconstruction for LDCT test set in the low-dose setting with Poisson-Gaussian noise. The pre-log noise model is given by $y_i = \\text{Poisson}(I_0 \\exp(-Ax_i)) + N(0, \\sigma^2)$. Following [6] to account for same dose level, we set $I_0 = 10^6$, $\\sigma = 5$. Results as below show that our method is robust to noise.\n|        |   coronal|  sagital | axial|\n|-----|----|-----|-----|\n| no noise | 37.87/0.968| 36.48/0.968| 35.69/0.966|\n|with noise| 37.62/0.966| 36.27/0.965 | 35.47/0.964|", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "kzJNWH1c4h", "reviewer": "Reviewer_FN1q", "summary": "This paper used the diffusion model to solve the widely known inverse problem in computed tomography. The paper is well-written and well-organized. \nI reckon this paper has two main contributions:\n1. The first diffusion model in CT considered z-axis consistency and used 3d data as neural network input. \n2. A SOTA results with astonishing visualization and metrics results.", "questions": "1. What is your window center and width for CT images in the main body of the paper? I think the current window width value you selected for the main part of the paper is quite wide, considering using a narrower one. I think some details are not so clear in this setting, though I can tell your result is better than others.", "limitations": "1. Although the author used some way to speed up the process both for training and sampling. However, diffusion models are still slow compared to other methods both traditional and deep learning and non-diffusion way.", "rating": 7, "confidence": 5, "soundness": 3, "presentation": 3, "contribution": 2, "strengths": "1. The first diffusion model in CT considered z-axis consistency and used 3d data as neural network input. \n2. A SOTA results with astonishing visualization and metrics results.", "weaknesses": "1. I admit this is the first paper I read using the diffusion model while considering z-axis consistency. However, the y-axis has been widely discussed and solved using various types of regularization between 2D slices or just using 3D data before. I consider the author just used a 3D way here to solve this point. Yeah, it is new for the diffusion model, but I do think it is less attractive to me considering the works already made before. \n2. The results achieved by the authors, particularly in sparse-view reconstruction using only four angles, are indeed remarkable. The quality of the reconstruction with such limited data is surprising and commendable. Given the significance of these results, it would be beneficial for the community if the authors consider sharing their code upon acceptance to enable further research and validation.\n3. The paper's novelty is somewhat overshadowed by the work \"Solving Inverse Problems in Medical Imaging with Score-Based Generative Models,\" which the authors have cited. It would be beneficial for the authors to clearly state the innovations and contributions of their work, especially in light of the foundational problem settings of inverse problem solving for medical imaging already addressed by the cited paper.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We thank the reviewer for providing the encouraging feedback! Below, we address the concern:\n\n> **Q:** *Discuss related work with other regularization methods*\n\nA: \nWe are aware of other works that apply external regularization between 2D slices with diffusion models such as [1,2] as we cited in our paper. Some other works train a xy plane prior and a yz plane prior [3] (also cited) for medical image reconstruction. Nevertheless, to the best of our knowledge, our work is the first one that uses only one 3D generative prior for medical image reconstruction without any additional regularization or priors. We will add more discussions on related works in our camera-ready version. \n\n> **Q:** *Release code*\n\nA: \nThank you for the encouraging feedback! We are in the process of refactoring our code, and will release it ASAP after acceptance.\n\n> **Q:** *Difference between the score-med paper*\n\nA: The problems we solve and our motivations are different from the score-med paper [4]. The score-med paper is one of the first works that leverages a 2D diffusion prior for solving 2D medical image reconstruction problems. Our work is the first work that investigates whether using a 3D patch diffusion prior can lead to improvement over a 2D diffusion prior for 3D medical image reconstruction. The answer to the question is positive as demonstrated in our paper. Also our baseline \"DDS 2D\" is a very similar approach to the score-med paper. We show that our method outperforms DDS 2D by a significant margin. \n\n\n> **Q:** *Window center and window width*\n\nA: We follow [1,2,3,4]'s approach for processing CT images, to see bone structures better. Our window center is 100 HU, and window length is 2800 HU ([-1300 HU, 1500 HU]). In the rebuttal pdf, we also provide the visualization of our reconstruction in a much narrower window [-150 HU, 250 HU], and we observe that our method can reconstruct the fine details and structures of images very accurately with more projections.\n\n\n> **Q:** *However, diffusion models are still slow compared to other methods both traditional and deep learning and non-diffusion way.*\n\nA: We demonstrate that our method and other diffusion-based approaches can be faster than TV-based iterative traditional approaches (SBTV (one of the best non-deep-learning approach)) in our rebuttal pdf. The reason is the computation of 3DTV can be costly, but our approach does not need any external regularization so it can be faster. \n\n\n\n[1] Chung, Hyungjin, et al. \"Solving 3d inverse problems using pre-trained 2d diffusion models.\" Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 2023\n\n[2] Chung, Hyungjin, Suhyeon Lee, and Jong Chul Ye. \"Decomposed Diffusion Sampler for Accelerating Large-Scale Inverse Problems.\" The Twelfth International Conference on Learning Representations.\n\n[3] Lee, Suhyeon, et al. \"Improving 3D imaging with pre-trained perpendicular 2D diffusion models.\" Proceedings of the IEEE/CVF International Conference on Computer Vision. 2023.\n\n[4] Song, Yang, et al. \"Solving Inverse Problems in Medical Imaging with Score-Based Generative Models.\" International Conference on Learning Representations.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:
Firstly, we would like to sincerely thank the reviewers for taking the time to review our paper and providing constructive feedback. We are encouraged that the reviewers think that in our paper 
- ```The first diffusion model in CT considered z-axis consistency and used 3d data as neural network input. A SOTA results with astonishing visualization and metrics results.``` (Reviewer  FN1q), 
- ```This paper proposes a novel method. Experimental results demonstrate that both methods are efficient and outperform previous works.``` (Reviewer YiqS), 
- ```The probabilistic modeling of the 3D volume taking into account neighboring slices is novel. The paper describes the author's motivation and implementation plan very well.``` (Reviewer Vnw7).

To address the concerns shared by some reviewers, we would like to globally highlight several key points in our response.

***1. This method outperforms the comparison method in PSNR/SSIM at fewer than 10 angles does not necessarily indicate better performance at other sparse angles, more traditional baseline results***

- We provide results on [20, 40, 60, 80, 100] views for our method (DiffusionBlend++) as well as baselines with those angles (including the SB(Split-Bregman)TV, CGLS, and SIRT baselines requested by Reviewer Vnw7) for the LDCT dataset in the rebuttal pdf and in the table below. Results show that our method ***outperforms the baselines significantly*** for ***each different number of angles*** we evaluated on. From Figure 1,2,3 in the rebuttal pdf, DiffusionBlend++ is able to reconstruct images very close to the ground truth with 20 projections or more, but other baselines such as SIRT and CGLS still struggle to get a satisfying reconstruction with 60 views or more. 
- The motivation of this paper is to exploit the 3D data-driven prior (instead of external cross-slice regularization or learning 2D priors on different planes) for medical image reconstruction. The hypothesis is that with the aid of the 3D prior, we can have better reconstruction quality compared to only using 2D priors. Many previous works leverage 2D diffusion priors for sparse-view reconstructions ( > 10 views), but only a few works exploit reconstructions with fewer than 10 views. By this consideration, we do not provide results for more than 10 projection in the main paper. Nevertheless, we will add additional results with more projection angles and more traditional baselines in the camera-ready version.


***2. A large number of structures that do not exist in CT images, which will interfere with the doctor's diagnosis***


- With ultra-sparse views (such as 4 views), CT reconstruction is extremely difficult since very few measurements are taken, so many traditional reconstruction algorithms completely fail in this scenario as demonstrated in Figure 3. Also, non-deep learning methods generate lots of structures that do not exist even with as high as 20 views as demonstrated in Figure 2 in the rebuttal pdf. To account for this missing information with ultra-sparse views, DiffusionBlend(++) generates many image structures that represent the ground truth image well, and some other structures that may look different from ground truth. Nevertheless, it is still significantly better than all baselines considered in this paper.
- With increasing number of views, more measurements are taken, and it is more likely we can reconstruct accurate structures (as mentioned by reviewer Vnw7). We provide reconstruction examples in the rebuttal pdf; results show that the image structures look ***almost identical*** to the ground truth with 40 views or more. 

***3. Difference from other related works***
- The problems we solve and our motivations are different from the score-med paper [1]. The score-med paper is one of the first works that leverages a 2D diffusion prior for solving ***2D*** medical image reconstruction problems. Our work is ***the first work*** that investigates whether using a 3D patch diffusion prior can lead to improvement over a 2D diffusion prior for ***3D*** medical image reconstruction. The answer  to the question is ***positive*** as demonstrated in our paper. Also our baseline "DDS 2D" is a very similar approach to the score-med paper. We show that our method outperforms DDS 2D by a significant margin. 
- Some other works use inter-slice external regularization, such as total variation (TV) [2] or multiple 2D priors on different planes for reconstruction [3], which we have both cited in our paper, as mentioned by reviewer FN1q. Other methods such as ADMM-TV uses the TV regularization for xy, xz, and yz plane. Nevertheless, our work differs from them fundamentally in that our work ***for the first time*** learns one generative 3D prior and uses only that prior without any other regularization for medical image reconstruction. We will include more discussions about related works in our camera-ready version.


***4. Discussion on Computational Efficiency***

We have provided the inference time of our method and baselines in Table 9 of our main paper.  In addition, we also provide training time, training memory, inference memory and inference time in the rebuttal pdf. Results show that we achieve better inference memory efficiency and decent inference speed compared to DDS since we do not use the computational-heavy total variation regularization (while DDS claims to be one of the most efficient diffusion solvers for medical imaging). ***Surprisingly***, we find the non-deep-learning method SBTV has a worse inference efficiency compared to our method due to the heavy computational cost of 3D total variation optimization.

[1] Song, Yang, et al. "Solving Inverse Problems in Medical Imaging with Score-Based Generative Models." ICLR2022

[2] Chung, Hyungjin, et al. "Solving 3d inverse problems using pre-trained 2d diffusion models." CVPR2023

[3] Lee, Suhyeon, et al. "Improving 3D imaging with pre-trained perpendicular 2D diffusion models."ICCV2023

Meta-review: Overall, all the reviewers are interested in the topics, have good opinions towards the proposed method and the experimental results, and have positive rating towards and tend to accept the paper. Most concerns from the reviewers have been well addressed by the authors to largely the satisfactions of the reviewers with more discussions, clarifications and experimental results.
Decision: Accept (poster)