DDGS-CT: Direction-Disentangled Gaussian Splatting for Realistic Volume Rendering
Authors: Zhongpai Gao
Benjamin Planche
Meng Zheng
Xiao Chen
Terrence Chen
Ziyan Wu
Venue: NeurIPS 2024 poster (2024)
Keywords: 3D Gaussian splatting
image registration
pose estimation
Primary area: machine_learning_for_healthcare
TL;DR: A novel approach that balances realistic physics-inspired X-ray simulation with efficient, differentiable DRR generation using 3D Gaussian splatting (3DGS)

Abstract: Digitally reconstructed radiographs (DRRs) are simulated 2D X-ray images generated from 3D CT volumes, widely used in preoperative settings but limited in intraoperative applications due to computational bottlenecks. Physics-based Monte Carlo simulations provide accurate representations but are extremely computationally intensity. Analytical DRR renderers are much more efficient, but at the price of ignoring anisotropic X-ray image formation phenomena such as Compton scattering. We propose a novel approach that balances realistic physics-inspired X-ray simulation with efficient, differentiable DRR generation using 3D Gaussian splatting (3DGS). Our direction-disentangled 3DGS (DDGS) method decomposes the radiosity contribution into isotropic and direction-dependent components, able to approximate complex anisotropic interactions without complex runtime simulations. Additionally, we adapt the 3DGS initialization to account for tomography data properties, enhancing accuracy and efficiency. Our method outperforms state-of-the-art techniques in image accuracy and inference speed, demonstrating its potential for intraoperative applications and inverse problems like pose registration.

DDGS-CT: Direction-Disentangled Gaussian Splatting for Realistic Volume Rendering
Zhongpai Gao'
Benjamin Planche" Terrence Chen
Meng Zheng
Xiao Chen
Ziyan Wu
United Imaging Intelligence, Boston, MA {first.. last) com
Abstract
Digitally reconstructed radiographs (DRRs) are simulated 2D X-ray images gen- erated from 3D CT volumes, widely used in preoperative settings but limited intraoperative applications due to computational bottlenecks, especially for curate but heavy physics-based Monte Carlo methods. While analytical DRR renderers offer greater efficiency, they overlook anisotropic X-ray image formation phenomena, such as Compton scattering. We present a novel approach that mar- ries realistic physics-inspired X-ray simulation with efficient, differentiable DRR generation using 3D Gaussian splatting (3DGS). Our direction-disentange (DDGS) method separates the radiosity contribution into isotropic and direction- dependent components, approximating complex anisotropic interactions without intricate runtime simulations. Additionally, we adapt the 3DGS initialization account for tomography data properties, enhancing accuracy and efficiency. Our method outperforms state-of-the-arr techniques in image accuracy. Furthermore, our DDGS shows promise for intraoperative applications and inverse problems such as pose registration, delivering superior registration accuracy and runtime performance compared to analytical DRR methods.
3DGS
Introduction
Motivation. Digitally reconstructed radiographs (DRRs) simulated (in silico) 2D X-ray images
rendered from 3D computational tomography (CT) volumes. While DRRs are widely utilized in preoperative settings, such optimizing dose delivery and in radiation oncology (20) 41 6), their potential intraoperative applications remain underexplored due to computational bottlenecks and naive modeling capability (7). For instance, real-time multimodal registration for image-guided procedures is currently impractical because of the time-consuming process of generating DRRs and integrating them into slice-to-volume registration (3)].
The advent of GPU-accelerated computing has significantly improved the efficiency of radiography though they still fall short of meeting the stringent requirements of real-time inverse graphics have led to the creation of differentiable
simulators B 39
applications. Recent developments DRR renderers (e.g.. DifIDRR [T], X-Gaussian [7]. GaSpCT (55),, which show promise for inverse
problems such as 2D/3D CT image registration 12 However, these methods do not fully capture computationally efficient ray-tracing techniques that model the attenuated photon fluence at each detector pixel by accumulating Hounsfield unit (HU) values along the 3D view-ray through the voxelized CT volume. This approach, while fast, cannot account for complex noise-inducing physical effects such as Compton scattering and beam hardening [14] crucial for accurate X-ray imaging.
the complexities of X-ray image formation. They typically
Equal contribution
38th Conference on Neural Information Processing Systems (NcurIPS 2024).

---

Tustmpic 3D Potnt
3DGS
Intrepprnitoe lrags
DOGS
CT veluree
Intolltatton
DDGS Direction Diseotaggled 3DGS)
Aciantopic SO Seint
Optrnaaiiz
DIESS Rendoee Inage
Proseradive
Introppnna vo 20:3D Registntio
Figure 1: Proposed pipeline of Direction-Disentaaeee 3D Gaussian Splatting (DDGS).
Physics- basses Monte- -arlo simulations on the decomposition of CT volumes into realistic materials via HU thresholding, they accurately simulate single-photon transport and probabilistically evaluate photon-matter interactions (photo- electric effect, Compton scattering, Rayleigh scattering, etc.) to determine the final attenuation and hit-region each photon. This approach provides a more accurate representation of X-ray image formation; but the computational intensity of these methods-requineee >10 iterations to render an image makes them impractical for real-time applications and inverse problems.
do not suffer from these limitations. Relying
In this paper, we propose novel, more elegant balance between (a) realistic physics-inspired X-ray simulation and (b) efficient, differentiable DRR generation, as illustrated in Figure
Contributions. Our approach leverages 3D Gaussian splatting (3DGS) 17 for efficient DRR generation from CT volumes, similar to X-Gaussian 7 and GaSpCT 25 However, we introduce a crucial adaptation: our of X-ray image formation on light directionality. Specifically, we decompose the radiosity contri- bution of 3D regions into two independent components: non-directional component modeling the isotropic X-ray interactions with the corresponding material, and a pirection-depentiee approximating gigher-diiensionnn generic vien-dependent ppherical-harmonee
3DGS (DDGS) method accounts for the dependencies
component anisotropic interactions, such scattering. Unlike the more decomposition adopted by common 3DGS solutions [10, ours is tuned to the specific behaviors of high-energy photons. This formulation allows method to implicitly learn and reproduce the effects of scattering on X-ray images without the need for complex photon-transport simulations at runtime.
Additionally, we adapt the initialization phase of 3DGS to account the properties of tomography data. We adopt geometry- and intensity-based dual strategy sample 3D points that correspond to: (a) contact regions between different anatomical elements, where photon transport likely to show discontinuities; and (b) homogeneous regions, where the primary attenuation contribution can be sparsely modeled but could benefit from view-dependent noise modeling.
We demonstrate that our solution achieves higher image accuracy compared to sfate-of-the-att methods across various medical benchmarks Additionally, we showcase the efficient application of our method to inverse problems, such pose registration, highlighting its potential for real-time intraoperative use cases. Our DDGS achieves better registration accuracy and runtime performance than analytical DRR methods.
2 Related Work
The generation of DRRs and the simulation of X-ray images have been subjects of extensive research due their critical roles medical imaging, particularly in radiotherapy planning and diagnostic imaging (20] 41 43). In this section, we provide some insight on X-ray imaging, traditional simulation tools, and the recent integration of machine-learinng techniques.
X-ray Image Simulation. X-ray imaging relies on the interactions of high-energy by the X-ray source of the scanner- with the matter (e.g., body part) placed in between the X-ray source of the scanner and its indiation-senstivee detector cells (dexels). Imaging contrast arises from

---

differential absorption and scattering of X-ray photons by tissues, primarily through the photoelectric effect, Compton scattering, and, lesser extent, beam hardening for polychromatic scanners. The photoelectric effect involves the absorption of an X-ray photon by an atom and is more pronounced in higher atomic number materials like bone, causing them to appear whiter on X-ray image whereas Compton scattering occurs when an X-ray photon collides with an outer electron, resulting in the photon being deflected and losing energy, which contributes to image noise and reduced contrast
[4 29
LS 3 model these X-ray interactions and light transport. They trace the paths of a large number of individual photons as they undergo probabilistic interactions within the material. By simulating numerous photon trajectories, Monte Carlo methods account for the random nature of photoelectric absorption and Compton scattering, as well as secondary effects such as fluorescence and electron transport. These MC solutions produce highly realistic simulations of radiographic images, though at high computational cost which makes them inadequate for most DRR use-cases. Furthermore, for these tools to render realistic DRRs from CT volumes, one must first pre-process the CT data to assign proper materials and their corresponding physical properties to each voxel, which essential for realistic simulation. This step is often approximated by decomposing the volume into small set of predefined materials according to user-provide eeer-provided HU ranges (e.g., "air" material assigned to voxels with HU value < -800, "lung" assigned to voxels in range - 800, -200]. "fat" for 200.- - 100), etc.). This process can be coneccinu and prone to errors, affecting the fidelity of MC-based DRR generation [9]
variety of Monte Carlo (MC) simulation suites 2
Efficient and Differentiable DRR Generation. Analytical DRR rendering methods 33 38 have been developed in parallel to the aforementioned MC suites for real-time or near-real time applications. These methods have a much lower computational footprint, they do not simulate individual photon interactions but instead use mathematical models to directly calculate the primary paths and approximated attenuations of X-rays. The seminal work of Siddon [1 computing line integrals through a discretized CT volume is still at the core of most recent DRR solutions, e.g., which proposed differentiable formulation of the rendering steps [1] or integrated neural networks to add realistic noise to the output [35]. However, the inherent approximations in these methods can affect their accuracy, particularly in modeling artifacts present in heterogeneous tissues or complex anatomical structures, e.g., caused by scattering (non-primary light contributions).
Recently, researchers 7 225 have tried to apply 3D Gaussian splatting (3DGS) techniques 17 to DRR rendering, ie.. optimizing a cloud of 3D Gaussians to approximate voxel data, thereby enabling faster rendering with minimal accuracy loss. However, these early solutions overlooked the specific nuances of X-ray imaging, directly applying 3DGS methods designed for natural imaging (25] or oversimplifying the physical properties of X-ray attenuation [25]. leading to suboptimal modeling.
In this paper, we propose novel formulation of 3D Gaussian splatting, tuned more efficiently approximate X-ray imaging (time- and quality-wise).
3 Methodology
3.1 Preliminaries
Gaussian Splatting. 3DGS [17 is point-based rendering technique using 3D Gaussians to explicitly represent scenes in a more compact manner than volumetric representations. Each Gaussian Ii [m,, E,, a,, fi} is defined by its mean position μ4; E R3, covariance matrix E, E 3xx3 (usually decomposed into separate scale and rotation parameters), opacity a; E R, and radiance properties the light contribution model adopted. E.g., k (L 1)2 x 3 for RGB radiance decomposed into spherical harmonics (SH) of degree L,
f: R where h depends anisotropic, view-depended model commonly used for natural-in nntural-imaging applications. Images rendered by casting view-rays through each pixel into the scene and alpha-blending the Gaussian contributions to the final ray color C, as:
C(p)= = coo
with a. a,e - 10- (p-ba) CoO (1- jEn
(I)
E=1
2X-ray film is historically white, turning black when exposed to high-energy photon.
3

---

where n is the number of Gaussians considered, ej is the radiance (computed from f,. e.g., via SH) of the jtth Gaussian on the ray, and B. and Σ, are the image-plane projections of μ4j and Eg.
Since the aforementioned rasterization process is differentiable w.r.t. to the Gaussian parameters, the 3DGS representation of scene can be learned via gradient-descent, given a set of 2D scene observations Er and their corresponding camera parameters. A combination of l and SSIM [40] loss functions are commonly-adopted as criterion for this iterative optimization process [7:
L= (1 - (TGT, I) + ASSMM (JGT: F),
(2)
with I the rendered images (based on provided camera parameters), and À loss weightiig hyper- parameter. The initial state is typically obtained by sampling relevant 3D points in the scene domain for the Gaussians, e.g., from
(SfM) point-cloud 334 28
Application to DRR. The volumetric rendering performed by 3DGS methods is conceptually similar the one performed by analytical DRR methods [I], i.e., aggregating the attenuation values of light-rays cast through the voxelized CT volume. This similarity has recently motivated researchers 7 25] to apply 3DGS models to DRR applications, driven by the compactness of 3DGS representations compared to voxel data (addressing the (n²²) complexity inherent to voxel rendering). By optimizing a 3DGS model approximate the visual properties of the CT volume, the resulting representation can render accurate DRRs much faster than traditional methods, making it more suitable for real-time intra-operative visualization. Consequently, the use of traditional, slower rendering solutions can be limited to generating DRR targets (IGT) for the offline (i.e., pre-operative) training.
25 do not account for noise- boise-inducing photon interactions (e.g., scattering) when applying the analytical methods 30 3 to render their training data. Since scatter-free DRRs are not affected by ray directions (i.e., the attenuation at each point is naively considered isotropic), Cai et al.
However, these early 3DGS-for-DRR solutions 7
7 simplified the radio-intensity function of each Gaussian to c: sigmoid (b- - fa), where b E R& is a direction-independenn optimizable basis vector shared by all splats. Furthermore, their Gaussian point-cloud is initialized via evenly-spaced Cai et al. ( or random-ue sondomm-niform (25) sampling of the scanned 3D space, i.e., ignoring relevant geometrical and material properties of the target CT data. Inadequate initialization strategies negatively impact the convergence of 3DGS models and the resulting image accuracy [16]. In this work, we propose to reformulate both the initialization strategy and the radiance function of the 3D Gaussians to address these limitations.
3.2 Disentanglement of Isotropic and Anisotropic 3D Gaussians
We argue that, while X-ray scattering models incompatible with the 3DGS rendering formulation (secondary rays would have to be cast, increasing the computational footprint exponentially), their anisotropic impact on X-ray imaging can be approximated to some extent by drrection-dependen radiance functions. However, modeling this high-dimensional residual contribution is only meaningful if it appears in the training DRRs, i.e., if the target radiographs are rendered using physics-inspired simulation tools. This is not always possible, e.g., if the mapping from HU values to materials for the target CT scans is not provided. Ideally, in such cases, when only scatter-free DRRs can be generated, the radiance function should gracefully degrade to lighter anisotropic formulation (e.g., similar to (7).. to avoid unnecessary computations that could impact the model optimization, as well as its runtime performance. Moreover, X-ray interactions with matter themselves involve both isotropic (e.g., photoelectric absooptioonftuoreseenee, Rayleigh scattering) and anisotropic (e.g., Compton scattering) phenomena [14] [2 Therefore, for the sake of both modularity and accuracy, we propose to decompose our DRR representation into isotropic Gaussians g!" and anisotropic. direction- dependent ones ofrr. We formulate their respective radiosity functions as:
ch sigmoid (biso f!50)
dir Sj
= sigmoid (O,.. Bdirgg!ii)
(3)
where fi f R are feature vectors respectively encoding the contribution of isotropic and anisotropic Gaussians, biso E IR global isotropic basis vector (similar to 77), and Bdir e Rk is second, anisotropic basis matrix that interacts with the L-degree spherical-deccomppsition Y : 2R² + Rez. applied to the ray angles 0, & without constant term (degree = 0), resulting in ky 22)

---

Anisomopic points
Isotropic points
Uniformly random
Evenly sample
Marching cubes
Density-weighed
Radiodensity-ewe ewarre (RA)
Figure 2: Illustration of the different sampling strategies for 3DGS initialization.
It is important to note that, while the usual spherical Aaherical-harmonics harmonics representation for Gaussian splatting also contains isotropic (degree 0) and anisotropic (degree 0) components, the model assumes that both contributions are co-located (i.e., they belong to the same 3D Gaussian Ji of position μ4; and covariance Σ.). account for the complexity of X-ray light transport, we relax this co-location constraint, i.e., modeling isotropic and anisotropic contributions via distinct 3D Gaussians (ght and of").. We demonstrate in our evaluation that this disentanglement results in both higher image quality and lighter representation.
3.3 Initialization via Radiodensity-Awar Dual Sampling (RADS)
Our second main contribution targets the initialization of the point-cloud that supports the Gaussian- mixture modeling of the 3D data. Since 3DGS was defined for natural imaging (e.g.. ignoring sub-surface light transport), common initialization strategies limit their sampling scene-surface points (SfM- 177 or depth-guided [24] subsampling). Such sampling of the 3D space is, however, inadequate for volumetric data, ignoring possibly salient regions. While uniform sampling of the CT space could be considered 7 25), we argue that such strategy is also suboptimal for anatomical data, discarding domain-relevan properties. We demonstrate empirically (c.f. Section that uniform sampling results in slower convergence of the 3DGS representation and degraded runtime performance.
Therefore, we propose a novel twofold sampling strategy that accounts for the specific nature of CT data and for our dual isotropic/directional 3DGS model. Based on the radiodensity values contained in the target CT volumes, our solution samples two set of 3D points pme E R x3 and pdw E X23 with distinct, complementary distributions (n1, n2 scalar hyper-paramette
Not unlike [24], the first set of points is sampled by applying the marching-cuber algorithm (1 extract points the interface between materials with different physical properties (i.e., distinet HU responses in the CT volume). These interface regions typically need careful modeling when simulating X-ray imaging, they result in discontinuous photon transport.
To complement this first set of points, the second set semi-randomly sampled from the voxel centroids, according to uniform distribution weighted by the voxels' radiodensities. l.e., voxels with higher radiodensity, contributing to the X-ray attenuation, are more likely to be picked to initialize the Gaussians. In other words, this rudiodensitywweighte strategy identifies additional points within the homogeneous regions the CT volume that also contributes to the imaging process.
Based on their distinct properties, we assign the entire pmce to initialize the isotropic Gaussians, whereas we equally split pow initialize both isotropic and direction-dependent Gaussians. All in all, our dual radiodensity-awan strategy improves the initialization of the 3DGS representation and more explainable than prior work (see Figure?
4 Experiments
4.1 Experimental Protocol
Implementation. For evaluation, we set the decomposition degree to L=1, the feature dimension to k=8, and initial cloud sizes =15,000 and =2=10,000. We apply loss weight X = 0.2. We choose the default threshold level (i.e., the average the minimum and maximum volume) for the marching-cuber algorithm. DDGS training is performed single NVIDIA A100 using Adam
5

---

Table 1: Comparison of Gaussian ppattiin-bbsse DRR rendering techniques, on 2 datasets.
3DGS 17
X-Gaussian
(k
32)
DDGS (Ours, 1.
8)
# points.
PSNR1
SSIM
points!
PSNR
SSIM
points!
PSNR
SSIM
abdomen
11,149
47.43
0.994
10,802
47.17
0.993
13,928
48.09
0.994
NAF-CT
chest
13,669
44.42
0.988
16.568
43.33
0.987
13,533
44.50
0.989
foot
8,616
44.51
0.984
10,909
44.34
0.985
10,786
44.70
0.985
jaw
17,902
40.47
0.973
22.318
40.02
0.972
19,665
40.57
0.974
avg
12,834
44.21
0.985
15,149
43.72
0.984
14,478
44.47
0.986
001
53,988
35.40
0.971
50,059
36.81
0.979
41, 41,947
37.88
0.984
002
49,099
37.03
0.982
48,113
37.79
0.986
43,933
38.43
0.988
CTPelvic JK: ServielK-Dattase Datasett
003
60,755
35.73
0.973
59,822
36.46
0.977
48,536
38.28
0.983
004
42,349
38.87
0.982
37,243
39.84
0.985
39,176
40.35
0.986
005
42,482
39.37
0.984
46,014
39.30
0.984
39,570
40.36
0.987
006
53,832
37.14
0.983
57,197
37.42
0.983
47,398
38,26
0.986
007
45,360
37.09
0.980
48,454
37.62
0.983
41,032
38.87
0.987
008
51,211
38.03
0,980
45,750
38.70
0,982
43,577
39.81
0.986
009
44,060
38,19
0.981
41,279
38.92
0.985
41,389
39.87
0.987
010
38,691
37.80
0.984
40,030
37.55
0.985
39,691
38.73
0.987
avg
48,183
37.47
0,980
47,396
38.04
0.983
42,625
39.08
0.986
optimizer [18] with learning rate of 1.25 x 10-4 for h and Bdir and 2.5 x 10-3 for fiso and fdir.
Default 3DGS [17 learning rates are applied the remaining parameters.
Datasets. We consider datasets. (1) NAF-CT 42 includes four CT images of abdomen, chest, foot, and jaw. For each, adopt TIGRE training and 50 randomly-distribuee ones for testing, in the range of [ 1-00,, 90%] (scatter-free DRRs). (2) CTPelvic1K [19] is large dataset of pelvic CT images. We consider the first 10 scans of the sub-dataset6 in our experiments. We adopt DeepDRR (35 to generate 60 training (evenly- and 60 caadooll-diitributedd testing DRRs (in the range [-60%, 60% D), taking advantage of the scattering-modeling capability of DeepDRR to render realistic X-ray data. (3) Ljubljana [26) is clinical dataset of 10 patients undergoing neurovascular surgery. Each patient underwent
to sample 50 evenly-distributed projections for
CT and two X-ray angiography seans. The pose each X-ray also provided. Following DiffPose strategy [12] randomly sample 900 projections for training and 100 for testing. (4) Finally, we provide some additional qualitative results on DeepFluoro 13 a collection of pelvic X-rays and CT images from six cadavers.
Metrics. We evaluate our method in two settings: novel-view synthesis and intraoperative 2D/3D image registration (i.e., pose estimation). For novel-view synthesis, SSIM as metrics. For intraoperative 2D/3D image registration, we consider the eтrоr both in terms of rotation (angular distance) and translation (Euclidean distance). We further measure the clinically- relevant registration accuracy of key anatomical landmarks for Ljubljana scans [26] where they are provided. Following 112] compute the target registration error (TRE) w.r.t. the positioning of these landmarks after registration.
the standard PSNR and
4.2 Novel-View Synthesis
We first validate our contributions in terms of the quality of rendered DRRs and compactness of our dual Gaussian-mixture representation.
Comparative Evaluation. We compare 3DGS 17 and X-Gaussian 7 on the NAF-CT 42 and CTPelvic K [19] datasets for the novel-view synthesis. For each method, we adopt the hyper- parameters recommended by their respective authors (e.g., L and for normal 3DGS [7 and
and 32 for X- X-Gaussian 77).
As shown in Table for NAF-CT 42 (without scatter-related noise), our DDGS outperforms both 3DGS and X-Gaussian terms of PSNR and SSIM. CTPelvicIK [19] (with scatter), our DDGS significantly outperforms 3DGS and X- X-Gaussian in both PSNR and SSIM while using considerably

---

Table 2: Novel-view synthesis evaluation on CTPelvic I data.
Steration=5000
2000
7000
15000
30000
PSNR
SSIM
PSNR
SSIM
PSNR
SSIM
PSNR
SSIM
PSNR
SSIM
DDGS (Ours)
001
25.77
0,908
30.46
0.944
35.18
0.975
36.91
0.981
38.04
0.984
(L 1, 8)
002
27.64
0.921
31.60
0.954
35.33
0.977
36.90
0.984
38.30
0.987
DDGS (Ours)
001
26.20
0.909
31.50
0.950
35.65
0.978
37.68
0.983
38.15
0.985
(I== 3. 8)
002
27.87
0.923
32.23
0.956
36.02
0.979
37.87
0.986
38.97
0.989
3DGS 17
001
23.56
0.873
27.52
0.914
31.66
0.951
63.64
0.963
35,36
0,970
(L
002
24.55
0,891
29.17
0.932
32.82
0.965
35.33
0.977
37.25
0.982
3DGS 17
001
23.55
0,872
27.54
0,914
33.34
0,962
36.01
0.975
37.12
0.980
(L= 3)
002
24.55
0.891
28.96
0.931
34.50
0.973
37.72
0.985
38.75
0.988
X- X-Gaussian 7
001
17.31
0.794
24.79
0.877
29.00
0.923
30.72
0.933
33.39
0,947
(k = 8)
002
13.45
0.742
27.19
0.904
32.72
0.962
34.52
0.973
36.04
0.978
X-Gaussian 7
001
26.34
0.854
31.28
0.950
33.56
0,968
35.07
0.974
36,83
0.979
(k = 32)
002
28.32
0,868
33.44
0.966
35,85
0.980
36.55
0.984
37.89
0.986
Table 3: Ablation study w.r.t. the proposed disentangled isottopiccaniisotopic representations.
DDGS (Ours)
direct-entangled PSNR
direct-independent PSNR
direct-dependent SSIM
3DGS-disentangled
PSNR
SSIM
SSIM
SSIM
PSNR
PSNR
SSTM
001
38.04
0.984
36.17
0.977
35.93
0.976
32.48
0,942
35.32
0.971
002
38.30
0.987
37.32
0.984
36.96
0.983
33.04
0.953
35.38
0.973
fewer points. Therefore, our DDGS method is more effective in simulating realistic X-ray images from CT scans.
Furthermore, we compare our DDGS (with degrees L = and L = 3 and feature dimension k: = 8) with traditional 3DGS (degrees L and L = 3) and X- Gaussian (feature dimensions k 8 and k = 32) on the 001 and 002 data of CTPelvic K [19] at iterations of 500, 2000, 7000, 15,000. and and k performs comparably to 3DGS with L = 3 and outperforms X- X-Gaussian with k = 32 after 15,000 iterations. Experiments on more data can be found in Table ST Our model with L 3 achieves the best performance overall. Figure 3 presents a qualitative comparison of the testing set, illustrating both the synthetic views and the generated 3D Gaussian points.
30,000, as shown in Table 2 Our model with
It should also be highlighted that, even though both the original CT scan (as voxel grid) and our 3DGS-based solution are explicit representations, the latter significantly more compact. E.g., our model can represent a CTPelvic K scan with only 42,625 Gaussians (c.f. Table defined by 19 float values each (3D position/rotation + 3D covariance + opacity feature vector) and a shared 32-dimensional basis vector b, so 809,907 float values in total; whereas the original CT scans are each composed of 512 512 x 500 93, 363, 200 values. Indeed, the voxel data may contain large homogeneous regions, which can be approximated with few Gaussians, hence a significant compression rate.
Finally, similar to [12] we qualitatively compare the results of recent DRR methods with actual X-ray images, for datasets providing both CT scans and real, posed projections (DeepFluoro [3] Ljubljana (26)). Results are shared in Figure with our method achieving state-of-the-art PSNR Further analyses provided in Appendix B
Ablation Study. Table demonstrates the effectiivnees of our prrection-disentaagee represen- tation for 3D Gaussians. For several settings, we observe performance degradation when learning the entangled), learning the the direetion-dependenn 3D Gaussians alone (direct deirect-eependert)... Additionally, we decouple the direction-dependen and direction-independeen components in 3DGS [17 (3DGGS-ixsentunlee)) which also showed lower performance compared to our learnable features. Figure dered views from disentangled 3D Gaussians, highlighting that the direction-dependentt 3D Gaussians
and direction-indeper independent components in the same 3D Gaussians (direct- 3D Gaussians alone (direct-ineependenn). or learning
5 shows the ren-
7

---

2,000
Iteration 7,000
Iteration
Ground truth
DIX0S1 (Ours)
190
42,261 points
41,912 points
27,233 points
45,522 points
52,915 points
30X3
38,467 points
45,981 points
50,059 points
X X-Gaunsin
Figure 3: Visualization of the Gaussian cloud optimization for different methods.
Real projection (GT)
DINDRR
3DGS
X-Gaussian
DDGS (Ours)
Predictions
Input CT
Signed error
Real projection (GT)
DINDRR
3DGS
X-Gaussian
DDGS (Ours)
Predictions
Input CT (SD)
PSNR 27.45
PSNR = 27.88
PSNR - 27.08
PSNR 28.48
Signed
Error legend:
Figure 4: Qualitative comparison of DRRs and real scans from DeepFluoro and Ljubljana datasets.
focus more on bones or anatomical structures, while the direction-inneependeet 3D Gaussians focus more the background.
Table showws the effectiveness of our proposed initialization scheme (radiodensity-awarr) compared to other initialization methods. Our scheme outperforms both the uniformly random sampling used in 3DGS [17 and the even sampling method used X-Gaussian (7].
At last, we conduct experiments the impact of feature dimension k. as shown in Table Our results indicate that increasing the feature dimension generally improves overall performance. Hlowevrr the improvement becomes marginal once the feature dimension exceeds k = 16.

---

Table 4: Impact of initialization strategies image quality during training, ie., measured CTPelvicIK scans w.r.. training iterations.
Iteration
Strategy
001
002
003
004
005
006
007
008
009
010
avg
RADS (ours)
25.86
27.72 25.24
28.52
30.52
28.31
29.23
27.92
28.95
29.20
28.15
500
Random
25.29
26.73
24.66
27.40
28.46 26.35 27.23 26.43
27.34
28.53
26.84
Even
24.96 26.64
24.40 26.60 27.54 26.63 25.06 26.01
26.15
26.92
26.09
RADS (ours)
35.02 37.06 36.47
36.49 36.79 34.52 35.56 36.03
36.44
35.55
35.99
7,000
Random
34.39 35.78
34.18
35.83 36.37 34.36 35.20 35.74
36.30
35.26
35.34
Even
34.22 36.07
33.65 36.31 36.64
34.13 34.97
35.85
35.61
35.33
35.28
RADS (ours)
37.88 38.43
38.28
40.35 40.36 38.26
39.81 39.87
38.73
39.08
30,000
Random
37.17
38.36
38.15 15 40.09 39.97 38.36 38.69
39.37
39.85
38.84
38.89
Even
37.33 38.51
37.83 39.58 39.12 38.27 38.87 39.88 39.46 38.95
38.78
Table Impact of & dimensionality on image quality, measured on 2 CTPelvic scans (001, 002).
k: 4
16
k 32
PSNR
SSIM
PSNR
SSIM
PSNR
SSIM
PSNR
SSIM
PSNR
SSIM
001
34.60
0.965
36.81
0.979
38.04
0.984
38.41
0.986
38.27
0.987
002
33.66
0.964
37.08
0.982
38.30
0.987
38.76
0.989
38.89
0.989
41,947 points
21,218 points
20,729 points
19,665 points
10,762 points
8,903 points
CTPelvic1K 001
jaN jaaN
30,606 points
24,521 points
6,085 points
4,379 points
3,313 points
1,066 points
DasFFluoo
550
gdr
unlyy
dir
only
iso
dir
g
Iso
only
dir
only
Figure 5: Visualization of the isotropic and anisotropic X-ray contributions to the rendered DRRs.
Target pose
Initial pose
Target initial
3DGS
Gaussion (k=B)
X-Gaussian (k=32)
DDGS (Ours
Figure Illustration of the registraittt convergence using various differentiable DRR renderers.
Table 6: Accuracy image registration on the CTPelvicIK [19] dataset.
3DGS 17
X-Caussian ?
(k=8)
X-Gaussian ? (A=32)
DDGS (Ours)
001
002
003
001
002
003
001
002
003
001
002
003
Rot. error (deg)
0.134
0.062
0.229
0.326
0.163
0.312
0.097
0.070
0.266
0.067
0.067
0.152
Trans. error (mm)
1.788
1.163
3.108
4.037
2.144
3.781
1.378
1.288
3.483
1.285
1.092
2.180

---

Table 7: Registration evaluation w.r.t. anatomical landmark accuracy and time efficiency, on Ljubljana.
DRR render time (ms) mean
total optimization time (s) 4
TRE (mm) med
Method
med
std
mean
med
std
mean
std
DiftDRR I DDGS
30.88
31.51
3.25
431.98
316.54 39.93
261.70
2.14
0.72
5.40
7.46
6.94
2.67
121.45
117.67
0.80
0.65
0.51
4.3
Application to Downstream Task (2D/3D CT Image Registration)
We validate our solution in terms of applicability to downstream tasks, considering here the registration of intraoperative 2D X-ray images to preoperative 3D CT volumes; a task erucial to a variety of clinical applications.
We choose the first five testing images on the 001, 002, and 003 CT data of CTPelvicIK IK [9 for image registration. Considering this task as an optimization-bans inverse graphics (i.e., rendering synthetic images based on predicted poses and comparing to the target image, then backpropagating the difference to improve the pose prediction), we adopt the framework from iComMA [32), replacing their image rendered by DDGS. We use Adam optimizer with learning rate of 0.05. We use the isocenter pose as the initialization pose. As shown in Table and Figure 6 our DDGS achieves the lowest rotation and translation errors.
problem that can be solved via
Finally, we also consider the experimental protocol proposed in DiffPose rely on a pretrained pose-regression CNN to get a first rough estimation of the scanner pose, before refining said pose via gradient-descent based optimization, leveraging their differentiable DRR renderer [.]. We adopt their posc-initialization network and testbed for the Ljubljana dataset, replacing their DiffDRR analytical renderer [1I by our DDGS, and compare to them in terms of runtime performance and baadmark-regitrrann accuracy. The results, reported in Table 7 highlight the convergence boost brought by our lighter solution.
12 where the authors
5 Conclusion
We proposed DDGS, a novel 3DGS- based representation for efficient and realistic DRR genera- tion from CT volumes. We extended traditional 3DGS by considering light directionality, crucial for accurate X-ray image modeling. By decomposing radiosity into non-directional and direction- dependent components, DDGS captures isotropic interactions and anisotropic effects. Unlike generic view-dependent aaherical-harmonee decomposition, DDGS implicitly learns scattering effects, sub- stantially improving the speed and accuracy of downstream clinical applications such as intraoperative 2D/3D registration.
Limitations. It should be noted that, similar to previous methods, our model expects an offline DRR renderer to produce realistic ground truth to train DDGS, which can be a limitation for some scenarios (e.g., when such renderer or certain parameters required for realistic rendering are not available). Our fireetion-dependen function relating to anisotropic X-ray effects may also fail to approximate complex multi-bounce light scenarios. Recent developments in N N-dimensional Gaussian splatting [8 show promising potential for modeling such cases. We should also note that the quality of DRRs is bound to the quality of the input CT scans. This issue affects both DRR methods and Monte-Carlo simulations (as the latter relies on the segmentation CT scans into waterials-speciff regions- if 3D scan is noisy, will the material segmentation and simulation results). When method uses simulators to get target images, it will face similar noise issues, though, our experiments show that DDGS barely introduces additional noise. Compensating for CT noise is an interesting, under-explored research direction that could benefit all DRR methods; but, we believe it is beyond the scope of our study.
The above limitations may restrict the adoption of DDGS to specific use cases, and our method should undergo clinical evaluation of the generated images in terms of anatomical accuracy. Nonetheless, we believe that our proposed method can positively impact both the computer vision and medical communitics by providing a more efficient and versatile DRR tool.
Societal Impact.
10

---

References
[1] John Adler Jr. Martin Murphy, Steven Chang, and Steven I. Hancock. Image-guided robotic radiosurgery. Neurosurgery, 44(6):1299-13000 1999.
[2] Sea Agostinelli, John Allison, K al Amako, John Apostolakis, H Araujo. Pedro Arce, Makoto Asai, D Axen, Swagato Banerjee, GJNI Barrand, al. Geant4 simulation toolkit. Nuclear instruments and methods in physics research section A: Accelerators, Spectrometers, Detectors and Associated Equipment, 506(3):250-300 2003.
[3] Andreu Badal and Aldo Badano. Accelerating monte carlo simulations of photon transport in voxelized geometry using massively parallel graphics processing unit. Medical physics, 36 (11):4878 4880, 2009.
[4] Julien Bert, Hector Perez-Ponce, Ziad El Bitar, Sébastien Jan, Yannick Boursier, Damien Vintache, Alain Bonissent, Christian Morel, David Brasse, and Dimitris Visvikis. Geant4-based monte carlo simulations on gpu for medical applications. Physics in Medicine & Biooloy, 58 (16):5593, 2013.
[5] Ander Biguri, Manjit Dosanjh, Steven Hancock, and Manuchehr Soleimani. Tigre: a matlab-gpi toolbox for cbet image reconstruction. Biomedical Physics & Engineering Express, 255)005001,, 2016.
[6] Mare A Bollet, Helen McNair, Vibeke N Hansen, Andrew Norman, Una O'Doherty, Helen Taylor, Mark Rose, Rahul Mukherjee, and Robert Huddart. Can digitally reconstructed radio- graphs (diTs) replace simulation films in prostate cancer conformal radiotherapy? International Journal of Radiation Oncology* Biology* Physics, 57(4): 122--1130 2003.
[7] Yuanhao Cai, Yixun Liang, Jiahao Wang, Angtian Wang, Yulun Zhang, Xiaokang Yang. Zong- wei Zhou, and Alan Yuille. Radiative gaussian splatting for efficient x-ray novel view synthesis. In European Conference on Computer Vision, pages 283-299. Springer, 2025.
[8] Stavros Diolatzis, Tobias Zirr, Alexandr Kuznetsov, Georgios Kopanas, and Anton Kaplanyan. N-dimensional gaussians for fitting of high dimensional functions. SIGGRAPH,, 2024.
[9] O Dorgham, MH Ryalat, and M Abu Naser. Automatic body segmentation for accelerated rendering of digitally reconstructed radiograph images. Informatics in Medicine Unlocked, 20: 100375, 2020.
[10] Sara Fridovich-Keie Alex Yu, Matthew Tancik, Qinhong Chen, Benjamin Recht, and Angjoo Kanazawa. Plenoxels: Radiance fields without neural networks. In Proceedings of the IEEE/CVF Conference Computer Vision and Pattern Recognition, pages 5501-5510, 2022.
[11] Vivek Gopalakrishnan and Polina Golland. Fast auto-differentiable digitally reconstructed radiographs for solving inverse problems in intraoperative imaging. In Workshop on Clinical Image-Based Procedures, pages 1-11. Springer, 2022.
[12] Vivek Gopalakrishhan, Neel Dey, and Polina Golland. Intraoperative 2d/3d image registration via differentiable x-ray rendering. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 1 11662-11672 2024.
[13] Robert B Grupp, Mathias Unberath, Cong Gao, Rachel A Hegeman, Ryan J Murphy, Clayton P Alexander, Yoshito Otake, Benjamin McArthur, Mehran Armand, and Russell Taylor. Automatic annotation of hip anatomy in fluoroscopy for robust and efficient 2d/3d registration. International journal of computer assisted radiology and surgery, 15:759-769, 2020.
[14] Bruce H Hasegawa. The physies of medical x-ray imaging. 1990.
[15] Sébastien Jan. G Santin, D Strul, Steven Staclens, K. Assié, D Autret, S Avner, R Barbier, M Bardies, PM Bloomfield, et al. Gate: a simulation toolkit for pet and spect. Physics in Medicine & Biology, 49(19 2004.
[16] Jaewoo Jung, Jisang Han, Honggyu An, Jiwon Kang, Seonghoon Park, and Seungryong Kim. Relaxing accurate initialization constraint for 3d gaussian splatting. arXiv preprint 203.0944 2024.
11

---

[17] Bernhard Kerbl, Georgios Kopanas. Thomas Leimkühler, and George Drettakis. 3d gaussian splatting for real-time radiance field rendering. ACM Transactions on Graphics, 1214 2023.
[18] Diederik P Kingma and Jimmy Ba. Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980, 2014.
[19] Pengbo Liu, Hu Han. Yuanqi Du, Heqin Zhu, Yinhao Li, Feng Gu, Honghu Xiao, Jun Li, Chunpeng Zhao, Li Xiao, al. Deep learning to segment pelvic bones: large-scale datasets and baseline models. International Journal of Computer Assisted Radiology and Surgery, 16: 749-756, 2021.
[20] Frank Lohr, Oliver Schramm, Peter Schraube, Gabriele Sroka-Perez, Steffen Seeber, Gerd Schlepple, Wolfgang Schlegel, and Michael Wannenmacher. Simulation of 3d-treatment plans in head and neck tumors aided by matching of digitally reconstructed radiographs (drr) and on-line distortion corrected simulator images. Radiotherapy and oncology, 45(2):199-25) 1997.
[21] William E Lorensen and Harvey E Cline. Marching cubes: A high resolution 3d surface construction algorithm. In Seminal graphics: pioneering efforts that shaped the field, pages 347-353, 1998.
http:://xbb.arccivv. org/web/ [22] Allessandro Migliori. X-ray fluorescence analysis. 2024-05-12.
Accessed:
[23] Ben Mildenhall, Pratul P Srinivasan, Matthew Tancik, Jonathan T Barron, Ravi Ramamoor- thi, and Ren Ng. Nerf: Representing scenes as neural radiance fields for view synthesis. Communications of the ACM, 65(1):99-106, 2021.
[24] Michael Niemeyer, Fabian Manhardt, Marie-Julie Rakotosaona, Michael Oechsle. Daniel Duckworth, Rama Gosula, Keisuke Tateno, John Bates, Dominík Kaeser, and Federico Tombari. Radsplat: Radiance field-informed gaussian splatting for robust real-time rendering with 900+ fps. arXiv preprini arXiv:2403. 13806, 2024.
[25] Emmanouil Nikolakakis, Utkarsh Gupta, Jonathan Vengosh, Justin Bui, and Razvan Mari- nescu. Gaspet: Gaussian splatting for novel ct projection view synthesis. arXiv preprint arXiv:2404.03 2024.
[26] Franjo Pernus et al. 3d-2d registration of cerebral angiograms: A method and evaluation on clinical images. IEEE transactions on medical imaging, 3250):1550-156 2013.
[27] Terry M Peters. Image- guide surgery: from x-rays to virtual reality. Computer methods in biomechanics and biomedical engineering. 4(1):27-57, 2001.
[28] Johannes L Schonberger and Jan-Michael Frahm. Structure-from soitie-froo-mootiioo revisited. In Proceed- ings of the IEEE conference on computer vision and pattern recognition, pages 4104-4113, 2016.
[29] J Anthony Seibert and John M Boone. X-ray imaging physics for nuclear medicine technologists. part 2: X-ray interactions and image formation. Journal of nuclear medicine technology, 33(1): 3-18, 2005.
[30] Gregory C Sharp, Rui Li, John Wolfgang, G Chen, Marta Peroni, Maria Francesca Spadea, Shinichro Mori, Junan Zhang, James Shackleford, and Nagarajan Kandasamy. Plastimatch: an open source software suite for radiotherapy image processing. In Proceedings of the XVI'th International Conference on the use of Computers Radiotherapy (ICCR), Amsterdam, Netherlands, volume 3, 2010.
[31] Robert I. Siddon. Fast calculation of the exact radiological path for a three-dimensional et array. Medical physics, 22(2):252-255, 1985.
12

---

[32] Yuan Sun, Xuan Wang, Yunfan Zhang, Jie Zhang, Caigui Jiang, Yu Guo, and Fei Wang. icomma: Inverting 3d gaussians splatting for camera pose estimation via comparing and matching. arXiv preprini arXiv:2312.09001, 2023.
[33] Andrea Tagliasacchi and Ben Mildenhall. Volume rendering digest (for nerf). arXiv preprint arXiv:2209.02417,222 2022.
[34] Shimon Ullman. The interpretation of structure from motion. Proceedings of the Royal Society of London. Series B. Biological Sciences, 203 (03(155))400-423 1979.
[35] Mathias Unberath, Jan-Nico Zaech, Sing Chun Lee, Bastian Bier, Javad Fotouhi, Mehran Armand, and Nassir Navab. Deepdrr-a catalyst for machine learning in fluoroscopy-guided procedures. In Medical Image Computing and Computer Assisted InterventionMICCAA 2018: s1 International Conference, Granada, Spain, September 16-20, 2018, Proceedings, Part IV 11, pages 98 106. Springer, 2018
[36] IMJ Van Der Bom, Stefan Klein, Marius Staring, R Homan, Lambertus W Bartels, and Josien PW Pluim. Evaluation of optimization methods for intensity-based 2d-3d registra- tion x-ray guided interventions. Medical Imaging 2011: Image Processing, volume 7962, pages 657-671. SPIE, 2011.
[37] Delio Vicini, Wenzel Jakob, and Anton Kaplanyan. A non-exponential transmittance model for volumetric scene representations. ACM Transactions on Graphics (TOG), 40(4):1-16, 2021.
[38] Franck P Vidal and Pierre-Frédéri Villard. Development and validation of real-time simulation of x-ray imaging with respiratory motion. Computerized Medical Imaging and Graphics, 49: 1-15, 2016.
[39] Franck Patrick Vidal, Manuel Garnier, Nicolas Freud, Jean-Michel Létang, and Nigel W John. Simulation of x-ray attenuation on the gpu. TPCG, pages 25-32, 2009.
[40] Zhou Wang, Alan C Bovik, Hamid R Sheikh, and Eero Simoncelli. Image quality assessment: from error visibility to structural similarity. IEEE transactions image processing. 13(4): 600-612, 2004.
[41] Charles Yang, Michael Guiney, Peter Hughes, Samuel Leung, Kuen Hoe Liew, James Matar, and George Quong. Use of digitally reconstructed radiographs in radiotherapy treatment planning and verification. Australasian Radiology, 44(():439-44) 2000.
[42] Ruyi Zha, Yanhao Zhang, and Hongdong Li. Naf: Neural attenuation fields for sparse-view cbet reconstruction. In International Conference on Medical Image Computing and Computer- Assisted Intervention, pages 442-452 452. Springer, 2022.
[43] Xinyang Zhang, Pengbo He, Yazhou Li, Xinguo Liu, Yuanyuan Ma, Guosheng Shen, Zhongying Dai, Hui Zhang, Weiqiang Chen, and Qiang Li. Dr-only carbon-ion radiotherapy treatment planning via deep learning. Physica Medica, 100: 100-128, 2022.
13

---

Supplementary Material
In this supplementary material, we provide further methodological context and showcase additional quantitative and qualitative results to highlight further the contributions claimed in the paper.
A
Methodological Insight
A.1 Transmittance in Natural vs. X-Ray Imaging
The exponential transmittance model T(t) used in NeRF and Gaussian-splattngg (GS) solutions for natural scenes-to describe the light attenuation as it travels through a medium from point r(fo) to r(t)-is based on Beer-Lambert law [6 23 33
T(t) = exp( - a(r(s))os).
(4)
where r(t) = o- is 3D point step on ray of origin and direction d, and a(p) is the medium's absorption/extinction (linked to its density) at point p. Assuming piece-wise homogeneous medium (with a, the absorption of homogeneous segment length 6;), this transmittance can be rewritten:
4 II (1 -1), 1=1
T(t) exp( Σ
as in our Equation 3.1 with 01 = 1 exp(- and 4 the end segment containing t. See 33 for proof.
X-ray attenuation from photoelectric effect (PE) follows the same model: at each step through the medium (e.g., voxel), the probability of photon extinction (i.e., averaged attenuation) is proportional to the atomic density. E.g., a ray travels through only voxels, and if it attenuated by 01 = 50% step 1 then 22 50% again step 2, then its total attenuation is 75% (c.f. (100 (100 30)) in accordance to Beer-Lambert.
Hence, Equation 3.1 (3DGS rendering) could be directly applied to DDR synthesis (assuming standard neglog scaling of absorption values into pixel ones [2 35 ), if we were to adopt a simplified model of X-ray imaging which only considers isotropic absorption (i.e., following the Beer-Lambert law) and optionally omit the term Cj E IRK (view-dependent radiance). This is the simplified model proposed in prior work [7].
In our paper, we propose to go beyond the isotropic simplification in X-Gaussian 7 and further account for anisotropic scattering of X-rays. This is why we preserve Cj and decompose it into two
terms: an isotropic term t and direction- dependent non-linear term ddir (Equation 3.2). Note that we also fix the dimension of these variables (i.e., the number of output channels) (monochromatic DRR) instead of three in natural 3DGS (RGB).
A.2 Joint Gaussian Rasterization
As explained in Section we define two distinct functions for the absorption contribution of the two Gaussian sets: one function is isotropic to approximate average audio-absorption (c)"), and one function is direction-depenndent to approximate the contribution of anisotropic Compton scattering to the image (cli") c.f. Equation
These two distinct Gaussian sets could be rasterized separately, as done in Figure 5 qualitative ablation. However, the two sets need to be rendered together to ensure correct simulation. Otherwise, Gaussians from one set occluded by others belonging to the other set could incorrectly contribute to the ray absorption. We refer the readers to Figure S1 for for an explanatory diagram. There, point y should not contribute to the final pixel value, as it is occluded by g (i.e., it absorbs all the remaining energy before the ray can reach gar). This could not be properly simulated if each Gaussian set rasterized separately.
14

---

g
g2 iso
dir
92
ray
absorption
91 dir
saturation
rendaring all
rendering g)
iso only
rendering g only
Figure S1: Importance of joint rasterization of Gaussian sets (simplified diagram).
By having 2 sets of Gaussians with distinct yet complementary contribution functions (isotropic versus direction-dependent functions), our solution can better model DRR imaging. During optimization, each Gaussian set will approximate its respective imaging effect, conditioned by its respective function.
We randomly split pmee (3D points sampled based on the CT scan distribution) into 2 sets, each subset contributing to the initialization positions of g and g (half the 3D points in pmo will serve as initialization for g:oo along with the entire pmee set; and the other half of pmee will serve as initialization for "fth). After initialization, during 3DGS-bass model optimization, the points each set of Gaussians move/split/drop independently (along with the optimization of their respective covariance/opacityfeennn values). Since the optimization of each set (isotropic and anisotropic) is conditioned their respective differentiable absorption function, they will acquire distinct properties.
B
Additional Experiments
B.1 Additional Comparison and Ablation Results
To further support the results in Table and Table 3 we evaluate method all the 10 selected data from the CTPelvicIK K dataset, as shown in Table ST
Table S1: Novel-view synthesis evaluation CTPelvic1 K data.
DDGS (ours) (L k 8)
3DGS
X-Gaussian (k 8)
DDGS (ours)
3DGS
X-Gaussian
direct-entangled Ablation Study
(L 1)
(L 3 8)
(L 3)
(k: = 32)
PNSR
SSIM
PNSR
SSIM
PNSR SSIM
PNSR
SSIM
PNSR SSIM
PNSR SSIM
PNSR
SSIM
001
38.04
.984
35.36
.970
33.39
.947
38.15
.985
37.12
.980
36.83
.979
36.17
.977
002
38.30
.987
37.25
.982
36.04
9778
38.97
.989
38.75
.988
37.89
.96
37.32
.984
003
38.21
.983
35.81
.974
33.99
39.42
.986
39.09
.984
35.84
.976
35.32
.975
40.54
38.43
.980
37.18
40.74
.986
39.06
39.70
005
39.88
.987
39.44
9985
37.50
.978
40.02
.987
40.68
.989
39.24
.985
39.36
.96
005
38.12
.986
36.69
.982
35.03
.977
38.79
.986
38.65
.987
37.14
.983
36.89
.983
007
38.91
.987
36.88
.978
35.51
.970
39.37
.988
38.58
.984
38.10
.987
37.72
.985
008
39.83
.979
36.70
40.19
39.70
.986
38.57
38.53
.982
009
39.67
.987
37.94
-980
37.60
.979
40.35
.988
39.16
.983
38.99
.987
38.74
.985
010
38.60
.987
37.50
.982
36.30
.978
39,18
.988
39.13
.986
37.89
.986
37.20
.985
avg
39.01
.986
37.32
.979
35.92
.972
39.52
.987
39.16
.985
37.96
.984
37.70
.983
B.2
Comparison Real X-ray Projections
The main purpose of DRRs is to provide quick visualization to clinicians (with the key metrics being speed and visibility), as well as to integrate larger imaging applications (with priority given again to speed and feature-level similarity w.r.t. real data). Therefore, prior DRR papers (traditional 30 31 or GS-based 7 (5)) mostly evaluate on downstream tasks (e.g., pose registration). few evaluate the image quality compared to other DRR tools (DifiDRR compares to Siddon's
15

---

Table S2: Quantitative comparison of analytical DRRs to real images from Ljubljana dataset.
SSIM med
PSNR med
Methods
mean
std
mean 1
std
Dif
.044
440
.179
24.16
24.35
2.82
3DGS 17
.409
.518
.199
22.57
23.44
2.78
X-Ga Gaussian 7
.409
.521
.198
22.58
23.42
2.78
DDGS (ours)
.459
.558
.204
25.61
27.35
3.48
and Plastimatch 30;; 7 to other NVS baselines). We found that only DiffPose (based on DifIDRR)
12
provides a qualitative comparison to some real scans.
Quantitative comparison to real data challenging. Not all imaging parameters are usually provided or accurate enough to create matching DRRs (w.rt. intensity, CT pose, etc.). It is especially hard to render DRRs aligned with real scans with existing Monte-Carlo (MC) simulation tools 12 15]: which likely why no DRR paper has performed such evaluation. Their interfaces and custom coordinate conventions are not compatible with pose annotations in public CT/X-ray datasets, and their documeentationn/uupoor lacking. Bridging the convention gap between MC and analytical communities would greatly benefit our domain and could be the focus of our next effort.
Nevertheless, do provide an indirect comparison to real projections. E.g., the registration experiments (Table/6) implicitly provide insight into the similarity between real and synthetic data, pose optimization is done by comparing DRRs real target scans.
In this supplementary. we share a more direct comparison. We use our DDGS-basee registration method to refine GT poses in Ljubljana data (c.f. aforementioned GT inaccuracies), then use the refined poses to render and compare DRRs to real scans. Results can be found in (ablddSaand Figure 4 E.g., when zooming into the images of the latter figure, observe that DiffDRR suffers from pixelization, unlike GS methods.
16

---

NeurIPS Paper Checklist
1. Claims
Question: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?
Answer: [Yes]
Justification:
The abstract and introduction explicitly list our contributions.
Guidelines:
The answer NA means that the abstract and introduction do not include the claims made in the paper.
The abstract and/or introduction should clearly state the claims made, including the contributions made in the paper and important assumptions and limitations. A No or
NA answer to this question will not perceived well by the reviewers.
The claims made should match theoretical and experimental results, and reflect how be expected to generalize other settings.
much the results
is fine to include aspirational goals as motivation as long is clear that these goals are not attained by the paper.
2. Limitations
Question: Does the paper discuss the limitations of the work performed by the authors?
Answer: [Yes]
Justification: Please refer to Section 5 in the main paper for detailed discussion.
Guidelines:
The answer NA means that the paper has no limitation while the answer No means that the paper has limitations, but those not discussed in the paper.
The authors are encouraged to create separate "Limitations" section in their paper.
The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g., independence assumptions, noiseless settings, model well-specification. asymptotic approximations only holding locally). The authors should reflect how these assumptions might be violated in practice and what the implications would be.
The authors should reflect on the scope of the claims made, e.g., if the approach was only tested on few datasets or with few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.
The authors should reflect the factors that influence the performance of the approach. For example, facial recognition algorithm may perform poorly when image resolution low images are taken in low lighting. Or speech-to-text system might be used reliably to provide closed captions for online lectures because it fails to handle technical jargon.
The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size.
If applicable, the authors should discuss possible limitations of their approach address problems of privacy and faimess.
While the authors might fear that complete honesty about limitations might be used by reviewers as grounds for rejection, a worse outcome might be that reviewers discover limitations that aren't 't acknowledged in the paper. The authors should use their best judgment and recognize that individual actions in favor of transparency play an impor- tant role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed to not penalize honesty concerning limitations.
3. Theory Assumptions and Proofs
Question:
For each theoretical result, does the paper provide the full set of assumptions and complete (and correct) proof?
Answer: [Yes]
17

---

Preliminaries are provided at the beginning of Section 3, and further theoretical background is provided in the supplementary material.
Justification:
Guidelines:
The answer NA means that the paper does not include theoretical results.
All the theorems, formulas, and proofs in the paper should be numbered and cross- referenced.
All assumptions should be clearly stated or referenced the statement of any theorems.
The proofs can either appear in the main paper or the supplemental material, but if they appear in the supplemental material, the authors are encouraged to provide short proof sketch provide intuition.
Inversely, any informal proof provided in the core of the paper should be complemented by formal proofs provided in appendix or supplemental material.
Theorems and Lemmas that the proof relies upon should be properly referenced.
4. Experimental Result Reproducibility
Does the paper fully disclose all the information needed to reproduce the main ex- perimental results of the paper to the extent that it affects the main claims and/or conclusions of the paper (regardless of whether the code and data are provided or not)?
Question:
Answer: [Yes]
Justification: Implementation and experimental details are provided, so that an expert in the art should be able to reproduce the presented results.
Guidelines:
The answer NA means that the paper does not include experiments.
If the paper includes experiments, a No answer to this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.
If the contribution is dataset and/or model, the authors should describe the steps taken make their results reproducible or verifiable.
Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is novel architecture, describing the architecture fully might suffice, or if the contribution a specific model and empirical evaluation, it may be necessary either make it possible for others to replicate the model with the same dataset, or provide access to the model. In general. releasing code and data is often one good way accomplish this, but reproducibility can also be provided via detailed instructions for how to replicate the results, access to a hosted model (e.g., in the case of large language model), releasing of a model checkpoint, or other means that appropriate to the research performed.
While NeurlPS does not require releasing code, the conference does require all submis- sions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example
(a) the contribution is primarily a new algorithm, the paper should make clear how reproduce that algorithm. (b) If the contribution is primarily new model architecture, the paper should describe the architecture clearly and fully.
(c) the contribution is a new model (e.g., a large language model), then there should either be way to access this model for reproducing the results or way to reproduce the model (e.g., with an open-source dataset instructions for how to construct the dataset).
(d) We recognize that reproducibility may be tricky in some cases, in which case authors are welcome describe the particular way they provide for reproducibility. In the case of closed-source models, it may be that access to the model is limited in some way (e.g., to registered users), but it should possible for other researchers to have some path to reproducing or verifying the results.
5. Open access to data and code
18

---

Does the paper provide open access to the data and code, with sufficient instruc- tions faithfully reproduce the main experimental results, as described in supplemental material?
Question:
Answer: [No]
Justification: All data used are public, and implementation release is pending approval at the moment.
Guidelines:
The answer NA means that paper does not include experiments requiring code.
Please see the NeurIPS code and data submission guidelines (https://nips... cc/
oliccy for more details.
While encourage the release of code and data, understand that this might not be possible, so "No" an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (e.g., for new open-source benchmark).
The instructions should contain the exact command and environment needed to run to reproduce the results. See the NeurlPS code and data submission guidelines (https
//nips.ccppblle pwwppp/guuuiieeeeep Sudi:///uuiieee/deeuuuubiiii SroodeSubiissi.e ssiniiolccyy for more details.
The authors should provide instructions data access and preparation, including how to access the raw data, preprocessed data, intermediate data, and generated data, etc.
The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. only subset of experiments are reproducible, they should state which ones are omitted from the script and why.
At submission time, preserve anonymity, the authors should release anonymized versions (if applicable).
Providing as much information as possible in supplemental material (appended to the paper) recommended, but including URLs to data and code permitted.
6. Experimental Setting/Details
Question: Does the paper specify all the training and test details (e.g., data splits, hyper- parameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?
Answer: [Yes]
Justification: See Section 4.
Guidelines:
The answer NA means that the paper does not include experiments.
The experimental setting should be presented in the core of the paper level of detail that is necessary to appreciate the results and make sense of them.
The full details can be provided either with the code, in appendix, as supplemental material.
7. Experiment Statistical Significance
Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?
Question:
Answer: [NA].
Justification:See Section
Guidelines:
The answer NA means that the paper does not include experiments.
The authors should answer "Yes" if the results are accompanied by error bars, confi- dence intervals, statistical significance tests, least for the experiments that support the main claims of the paper.
The factors of variability that the error bars are capturing should be clearly stated (for
example, train/test split, initialization, random drawing of some parameter, or overall with given experimental conditions).
19

---

The method for calculating the error bars should be explained (closed form formula, call to a library function, bootstrap, etc.)
The assumptions made should be given (e.g.. Normally distributed errors).
should be clear whether the error bar is the standard deviation or the standard error of the mean.
It is OK to report 1-sigma error bars, but one should state it. The authors should preferably report a 2-sigma error bar than state that they have 96% CI, if the hypothesis of Normality of errors is not verified.
For asymmetric distributions, the authors should be careful not to show in tables figures symmetric error bars that would yield results that are out range (c.g. negative emror rates).
If error bars reported in tables plots, The authors should explain in the text how they were calculated and reference the corresponding figures or tables the text.
8. Experiments Compute Resources
For each experiment, does the paper provide sufficient information on the com- puter resources (type of compute workers, memory, time of execution) needed reproduce the experiments?
Question:
Answer: [Yes]
Justification:
See implementation details in Section 4.
Guidelines:
The answer NA means that the paper does not include experiments.
The paper should indicate the type of compute workers CPU or GPU, internal cluster, cloud provider, including relevant memory and storage.
The paper should provide the amount of compute required for each of the individual experimental runs as well as estimate the total compute.
The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g., preliminary or failed experiments that didn't 't make it into the paper).
9. Code Of Ethics
Question: Does the research conducted in the paper conform, in every respect, with the NeurIPS Code of Ethics
Answer: [Yes]
Justification: All standard guidelines were followed.
Guidelines:
The answer NA means that the authors have reviewed the NeurIPS Code of Ethics.
If the authors answer No, they should explain the special cireumstances that require deviation from the Code of Ethics.
The authors should make sure to preserve anonymity (e.g., if there is special consid- eration due to laws or regulations in their jurisdiction).
10. Broader Impacts
Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?
Answer: [Yes]
Justification: Please refer to Section 5 of the main paper for further details.
Guidelines:
The answer NA means that there is no societal impact of the work performed.
If the authors answer NA or No, they should explain why their work has no societal impact or why the paper does not address societal impact.
Examples of negative societal impacts include potential malicious or unintended uses (e.g., disinformation, generating fake profiles, surveillance), fairness considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.
20

---

The conference expects that many papers will be foundational research and not tied particular applications, let alone deployments. However, if there is direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that an improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster.
The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology being used intended but gives incorrect results, and harms following from (intentional or unintentional) misuse of the technology.
If there negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition to attacks, mechanisms for monitoring misuse, mechanisms monitor how system learns from feedback over time, improving the efficiency and accessibility of MI..
11. Safeguards
Does the paper describe safeguards that have been put in place for responsible release of data models that have high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?
Question:
Answer: [NA]
Justification: [NA]
Guidelines:
The answer NA means that the paper poses such risks.
Released models that have high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere usage guidelines or restrictions to access the model or implementing safety filters.
Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.
We recognize that providing effective safeguards is challenging, and many papers do not require this, but we encourage authors to take this into account and make best faith effort.
12. Lieenses for existing assets
Are the creators or original owners of assets (e.g., code, data, models), used in the paper. properly credited and are the license and terms of use explicitly mentioned and properly respected?
Question:
Answer: [Yes]
Justification: Public datasets and libraries are referenced.
Guidelines:
The answer NA means that the paper does not existing assets.
The authors should cite the original paper that produced the code package or dataset.
The authors should state which version of the asset is used and, if possible, include URL.
The name of the license (c.g., CC-BY 4.0) should be included for each asset.
For scraped data from a particular source (c.g., website), the copyright and terms of service of that source should be provided.
If assets are released, the license, copyright information, and terms of use in the package should be provided. For popular datasets, paperswi thcode. com com/datasets has curated licenses for some datasets. Their licensing guide can help determine the license of dataset.
For existing datasets that are re-packaged, both the original license and the license of the derived asset (if it has changed) should be provided.
21

---

If this information is not available online, the authors are encouraged to reach out to the asset's 's creators.
13. New Assets
Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?
Answer: [NA]
Justification: [NA]
Guidelines:
The answer NA means that the paper does not release new assets,
Researchers should communicate the details of the dataset/code/moodl as part of their submissions via structured templates. This includes details about training, license, limitations, etc.
The paper should discuss whether and how consent was obtained from people whose asset is used.
At submission time, remember to anonymize your assets (if applicable). You can either create an anonymized URL or include an anonymized zip file.
14. Crowdsourcing and Research with Human Subjects
Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well details about compensation (if any)?
Answer: [NA]
Justification: [NA]
Guidelines:
The answer NA means that the paper does not involve crowdsoureing nor research with human subjects.
Including this information in the supplemental material is fine, but if the main contribu- tion of the paper involves human subjects, then as much detail as possible should be included in the main paper.
According the NeurIPS Code of Ethies, workers involved in data collection, curation, or other labor should be paid at least the minimum wage in the country of the data collector.
15. Institutional Review Board (IRB) Approvals or Equivalent for Research with Human Subjects
Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or an equivalent approval/review based on the requirements of your country or institution) were obtained?
Answer: [NA]
Justification: [NA]
Guidelines:
The answer NA means that the paper does involve crowdsourcing research with human subjects.
Depending on the country in which research is conducted, IRB approval (or equivalent) may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.
We recognize that the procedures for this may vary significantly between institutions and locations, and we expect authors to adhere to the NeurlPS Code of Ethics and the guidelines for their institution.
For initial submissions, do not include any information that would break anonymity (if applicable), such as the institution conducting the review.
22


Reviews:
[{"review_id": "Zf9CMMBajK", "reviewer": "Reviewer_Zg8U", "summary": "This manuscript presents a novel method called Direction-Disentangled Gaussian Splatting (DDGS-CT), tailored for balancing realistic X-ray simulation and efficient DRR generation using 3D Gaussian Splatting (3DGS). It addresses the challenges posed by intricate physics computation, which often harms the application of Monte Carlo simulations. The method decomposes the radiosity into isotropic and direction-dependent components to approximate complex anisotropic interactions without runtime simulation. Notably, it achieves significant improvements in PSNR and SSIM on DRR renderings of multiple organs, demonstrating superior performance.", "questions": "1.\tI wonder if it is necessary to disentangle the isotropic and anisotropic point sets since they are both splatted to render the output images. I think if they are disentangled, it would be more reasonable that they are also rendered using different algorithms.\n2.\tAs far as I know, 3DGS is actually an explicit representation of 3D data. I wonder if it is necessary to transform 3D data from an explicit representation (volume) into another explicit representation (3DGS).", "limitations": "The authors have adequately discussed their limitations.", "rating": 5, "confidence": 4, "soundness": 3, "presentation": 2, "contribution": 3, "strengths": "1.\tInnovative Approach: This manuscript introduces an innovative method of X-ray simulation techniques to balance the efficiency and realism in DRR rendering, which is a significant step forward in the field of medical imaging.\n2.\tSignificant Empirical Improvements: The method substantially improves PSNR and SSIM scores, demonstrating its effectiveness over existing methods.\n3.\tDetailed Methodological Framework: This manuscript presents a well-structured and comprehensive methodological framework, introducing the disentanglement of isotropic and direction-dependent components and initialization strategy that accounts for the nature of CT data.", "weaknesses": "1.  “… we equally split P^dw to initialize both isotropic and direction-dependent Gaussians.” I’m confused about how to split the points set P^dw into disentangled isotropic and anisotropic point sets. The authors should provide a more detailed explanation.\n\t2. How do the disentangled isotropic and anisotropic 3DGS be rendered? Based on Figure 1, it seems that the two disentangled 3DGSs are integrated to render the output. If so, why should they be disentangled? The authors should explain the motivation for this design.\n\t3. In the Experiments, the 3D Gaussian-based methods are evaluated, yet the DRR-based methods mentioned above are not validated both in qualitative and quantitative comparison. Can they be added?", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "### Isotropic vs. anisotropic sets — motivation, initialization, and rendering:\nWe hope that the following points will clarify some misunderstandings:\n- We define 2 different functions for the absorption contribution of the 2 Gaussian sets: one function is isotropic to approximate average radio-absorption ($c_i^{\\text{iso}}$), and one function is direction-dependent to approximate the contribution of anisotropic Compton scattering to the image ($c_j^{\\text{dir}}$).\n- Since we have 2 distinct Gaussian sets, they could indeed be rasterized separately, as done in Figure 4 purely for qualitative ablation. However, the two sets need to be **rendered together to ensure correct simulation**. Otherwise, Gaussians _occluded_ by others belonging to the other set could incorrectly contribute to the ray absorption. Please see Figure R2 in rebuttal PDF for an example: there, point $g_2^{\\text{dir}}$ should not contribute to the final pixel value, as it is _occluded_ by $g_2^{\\text{iso}}$ (i.e., it absorbs all the remaining energy before the ray can reach $g_2^{\\text{dir}}$). This could not be properly simulated if each Gaussian set is rasterized separately.\n- By having 2 sets of Gaussians with distinct yet complementary contribution functions (isotropic vs. direction-dependent functions), our solution can better model DRR imaging. During optimization, each Gaussian set will approximate its respective imaging effect, conditioned by its respective function.\n- We randomly split $\\mathbf{P}^{\\text{mc}}$ (3D points sampled based on the CT scan's distribution) into 2 sets, each subset contributing to the initialization positions of $g_i^{\\text{iso}}$ and $g_j^{\\text{dir}}$ (half the 3D points in $\\mathbf{P}^{\\text{mc}}$ will serve as initialization for $g_i^{\\text{iso}}$, along with the entire $\\mathbf{P}^{\\text{mc}}$ set; and the other half of $\\mathbf{P}^{\\text{mc}}$ will serve as initialization for $g_j^{\\text{dir}}$). After initialization, during 3DGS-based model optimization, the **points in each set of Gaussians can move/split/drop independently** (along with the optimization of their respective covariance/opacity/feature values). Since the optimization of each set (isotropic and anisotropic) is conditioned on their respective differentiable absorption function, they will acquire distinct properties.\n\nWe will add more details to the paper and will provide the new figure in the annex of the camera-ready version.\n\n### Comparison to \"DRR-based methods\":\nSince our method belongs to the family of _analytical_ DRR rendering methods [7, 11, 29, 30, 24] and to the more recent sub-family of 3DGS-based solutions [7, 24], we evaluated it accordingly. As standard in this domain [5, 7, 11, 12, 33] (see discussion with Reviewer `g5Vp`), we compared on meaningful downstream tasks (CT/X-ray pose estimation and 3D/2D keypoint registration) with corresponding SOTA methods [11, 12, 7] and baselines [17] (note that we did not consider [24], as its contributions w.r.t. [17] are marginal). Additionally, though less common in the domain of analytical DRR rendering (e.g., [7, 11] only compare to a few methods), we also provided an image-quality evaluation over several datasets, comparing to Monte-Carlo (MC) simulation results (used as GT, which is standard in DRR evaluation [5, 7, 11, 33]). Based on a suggestion by Reviewer `g5Vp`, we further provide some quantitative comparison to real projections (see rebuttal PDF).\n\nTherefore, by **combining image-quality evaluation** (comparing to MC and real data) **and downstream-task evaluation on multiple datasets**, against several relevant SOTA analytical methods, we believe that we have been more thorough than most DRR papers. \n\n### Voxel vs. Gaussian representations:\nThe reviewer is correct that both the original CT scan representation (voxel grid) and the proposed 3DGS-based one are explicit. However, as shown in our experiments, our 3DGS representation is **much more compact**. E.g., for the CTPelvic1K data, our model can represent a CT scan with only ~42,625 Gaussians (c.f. Table 1), defined by 19 float values each (3D position/rotation + 3D covariance + opacity + feature vector) and a shared 32-dimensional basis vector $\\mathbf{b}$, so $809,907$ float values in total; whereas the original CT scans are each composed of $512\\times 512\\times 500=93,363,200$ values. E.g., the voxel data can contain large homogeneous regions which can be approximated with few Gaussians, resulting in a **significant compression rate**.\n\nMoreover, the original voxel grids contain raw absorption values, which need further heavy processing to be projected into DRRs (volume traversal and rendering). In comparison, the 3DGS representation is already meant for fast rendering, i.e., facilitating the rasterization process. For the sake of DRR generation, we demonstrate (e.g., in Table 7, against DiffDRR [11, 12] which relies on voxel representation) that 3DGS models are **much lighter, both in terms of forward rendering and backward propagation** (less operations), making them better suited for integration into larger automation systems.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "lqhsAktXOE", "reviewer": "Reviewer_YFhr", "summary": "This paper proposes DDGS, a Gaussian splatting (GS) based method for rendering realistic 2D X-ray images from 3D CT volumes. Taking advantage of GS, the proposed method operates in real-time. Moreover, the DDGS model employs isotropic Gaussians and anisotropic, direction-dependent Gaussians to model complex X-ray physical interactions, such as Compton scattering. The proposed method is well-motivated and technically sound. The effectiveness of the DDGS model is confirmed on multiple datasets.", "questions": "See the section of Strengths and Weaknesses, please.", "limitations": "The authors mention enough limitations of their work.", "rating": 6, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "**Motivation**\n\nConventional Monte-Carlo methods can produce realistic X-ray images, but they are time-consuming. Existing real-time GS-based methods ignore complex X-ray interactions, limiting reconstruction performance. The proposed DDGS model integrates complex X-ray physical interactions into GS to achieve real-time, high-quality X-ray image rendering. This is well-motivated.\n\n**Technical novelty**\n\nThe proposed DDGS model introduces novel anisotropic, direction-dependent Gaussians to account for the Compton scattering effect. This is a good idea and technically sound. Moreover, the authors propose a novel Radiodensity-Aware Dual Sampling method for CT volumes.\n\n**Clarity and organization**\n\nThis paper is well-written and easy to follow.\n\n**Experimental evaluation**\n\nThe authors performed extensive experiments on three different datasets, including comparisons with state-of-the-art (SOTA) methods, ablation studies, and downstream tasks. The results confirm the effectiveness of the DDGS model.", "weaknesses": "**Effectiveness of the Radiodensity-Aware Dual Sampling**\n\nFrom Table 4, the improvements made by the Radiodensity-Aware Dual Sampling are marginal (<0.3 dB), so its effectiveness needs further investigation.\n\n**Lack of CT volumes**\n\nAlthough the purpose of this work is digitally reconstructed radiographs, showing the CT volume represented by GS is necessary for evaluating the proposed model.\n\n**Visualization results**\n\nThe qualitative results shown in Figure 3 are hard to distinguish visually. Please consider showing the error maps between the reconstructions and the ground truth.\n\n**Some typos**\n\nFor example, in line 90, “] − 800, −200], “fat” for ] − 200, −100]” should be “[ − 800, −200], “fat” for [ − 200, −100]”.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "### Effectiveness of the radiodensity-aware dual sampling:\nWe believe that the accuracy increase compared to the SOTA [7] (i.e., performing uniform sampling) brought by this single contribution (novel radiodensity-based sampling) is still significant.\n\nWe should further highlight that our novel CT-specific initialization scheme **significantly improves the convergence of 3DGS models** for DRR rendering, **as shown in the new Table R3** (see rebuttal PDF). E.g., after 500 iterations, our model with the proposed sampling strategy outperforms the one with SOTA even-sampling [7] by +2.06dB, and the one with random-sampling [24] by +1.31dB. The gap slowly reduces as models converge and reach saturation in terms of image quality; but this shows that our contribution facilitates convergence. We will add this insight to the manuscript.\n\n### CT volume visualization:\nWe thank the reviewer for their insightful suggestion. We will add some volumetric visualizations of input CT scans to our qualitative figures. Please refer to the rebuttal Figure R1 (in the attached PDF) to see how they would look. Note that we used the standard medical imaging platform Slicer [r6] to generate these CT visualizations. \n\n### Error map visualization:\nWe thank the reviewer for another valid suggestion. We will add signed error maps to Figure 3 (we find signed error maps to be more insightful than error maps for DRRs, as they highlight where the methods may underestimate or overestimate the overall photon absorption). An example of how these error maps would look **can be found in the new Figure R1** (in the attached PDF), where we compared DRR outputs to real X-ray scans (c.f. discussion with Reviewer `g5Vp`).\n\n### Typos:\nThank you, we will further proofread our manuscript before final submission. \n\nFor the mentioned typo example, however, we would like to highlight that $]a, b]$ and $(a, b]$ are both valid, equivalent, notations for the left-open interval $a < x \\leq b$ (c.f. standard ISO 31-11 for mathematical notations). We understand that the second notation may be more prevalent in our domain and will change the paper accordingly.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "MRO1vxahvb", "reviewer": "Reviewer_ZGY5", "summary": "Digitally reconstructed radiographs (DRRs) are simulated 2D X-ray images generated from 3D CT volumes, widely used in preoperative settings but limited in intraoperative applications due to computational bottlenecks. In this paper, the author proposes an approach that combines realistic physics-inspired X-ray simulation with efficient, differentiable DRR generation using 3D Gaussian splatting (3DGS). This method excels in both image accuracy and inference speed, demonstrating its potential for intraoperative applications and inverse problems like pose registration.", "questions": "1. The methods section lacks explanations for some symbols. Please review and provide explanations where necessary. Additionally, are g_i^{iso} and c_i^{iso}, as well as g_i^{dir} and c_i^{dir}, referring to the same concepts? Please review carefully. For the methods section, it is recommended to include an overview diagram for clarification.\n2. Add more true sota methods for comparison.", "limitations": "1. The experiments are insufficient. The comparison method lacks representativeness and fails to demonstrate the true level of the field.\n2. The method description is confusing and the contributions are not clear.", "rating": 3, "confidence": 4, "soundness": 2, "presentation": 2, "contribution": 2, "strengths": "Digitally reconstructed radiographs (DRRs) are simulated 2D X-ray images generated from 3D CT volumes, widely used in preoperative settings but limited in intraoperative applications due to computational bottlenecks. In this paper, the author proposes an approach that combines realistic physics-inspired X-ray simulation with efficient, differentiable DRR generation using 3D Gaussian splatting (3DGS). This method excels in both image accuracy and inference speed, demonstrating its potential for intraoperative applications and inverse problems like pose registration.", "weaknesses": "1. The methods used for comparison are limited and do not reflect the current state of the field. The state-of-the-art methods claimed by the author are not widely accepted as state-of-the-art by researchers, and some have not been peer-reviewed, like reference [7,11]. Therefore, I believe the experimental results lack persuasiveness.\n2. The structure of the paper is not well organized. Despite the author clearly stating the motivation and contributions, the intentions of the author remain unclear. I have a rough idea of what the related work section is trying to convey, but its organization is also poor. It is recommended to include the 'Application' from section 3.1 within the related work.\n3. In the ablation experiments, only a portion of the data was tested. Results from testing the entire dataset would be more convincing.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "### SOTA status of X-Gaussian and DiffDRR/DiffPose:\nWe respectfully disagree with the reviewer w.r.t. where the SOTA stands. Since we started writing this paper, DiffPose [12] (an extension of DiffDRR [11] by the same authors) has been presented at **CVPR 2024 (_oral_)**, and X-Gaussian [7] has been approved at **ECCV 2024**. We believe that this demonstrates that our analysis of the state of the field is correct and that our **comparison to these methods is essential**. Note that, at a fundamental level, we do agree that the current trend in machine learning to consider preprints as legitimate work can be counterproductive to science's steady progress. In this case, however, we made a decision, now validated by peers, based on the quality of these works [7, 11, 12] and our knowledge of the field.\n\nThe application of novel inverse graphics techniques (e.g., 3DGS) to DRR rendering is a fast-moving field, but we did our best to reference the relevant methods. We especially consider X-Gaussian [7] and DiffDRR/DiffPose [11, 12] to be the most important for comparison, as they **each represents the state-of-the-art in their respective type of analytical DRR rendering** (X-Gaussian for 3DGS-based DRR rendering, and DiffPose as a differentiable extension of Siddon's voxel-based algorithm [30] enabling optimization-based tasks), directly relevant to our work.\n\n### Manuscript organization:\nWe thank the reviewer for finding our motivation and contributions clear, and for the suggestion to shuffle a bit the Related Work and Preliminaries sections. We agree that our manuscript may benefit from moving some application-related sentences to the former section.\n\nRegarding the suggestion to express our _intentions_ as clearly as our _motivation_:\nour key goal is to define a DRR rendering solution that (a) is fast and differentiable in order to enable intraoperative tasks (e.g., CT/X-ray registration); but also that (b) matches more closely the complex X-ray imaging process without sacrificing performance. Our intuition is that a better trade-off between realism and computational performance (compared to recent 3DGS-based solutions for DRRs, which ignore anisotropic properties) could further help downstream tasks, as demonstrated in the paper (accurate MC simulation approximation and pose estimation).\n\n### Additional ablation results:\nWe understand that performing some of the ablation studies on data subsets is not ideal. We believed that the selected instances were statistically representative enough and wanted to preserve space and computing power for the other quantitative results. It is not ideal, but also common with rich 3D datasets to perform ablation studies on a subset [8, 17, 31, 35, 36, etc.].\n\nWe **complete our ablation studies**, with results shared **in Tables R2-R3** (in the rebuttal PDF). More will be added to the paper and supplementary material.\n\n### Symbol explanations:\nCould the reviewer specify which unexplained symbols they are referring to, please? Note that we went through our manuscript and found 1 typo, L208: we meant to write \"loss weight $\\lambda$\" rather than \"loss weight $\\delta$\" (referring to the symbol in Equation 2). We thank the reviewer for helping us correct this.\n\nRegarding $g_i^{\\text{iso}}$ and $c_i^{\\text{iso}}$ (as well as $g_j^{\\text{dir}}$ and $c_j^{\\text{dir}}$), those are different concepts (L163-165). We used the notation $g$ to represent Gaussians (each defined by its 3D mean position, covariance, opacity, and color/intensity features c.f. L113). So $g_i^{\\text{iso}}$ and $g_j^{\\text{dir}}$ each represents a distinct set of Gaussians, with different 3D positions, covariances, opacities, and, more importantly, different feature vectors (resp. isotropic $\\mathbf{f}_i^{\\text{iso}}$ and direction-dependent $\\mathbf{f}_j^{\\text{dir}}$). \n\nIn contrast, we used the symbol $c$ in the paper to represent the color/absorption contribution of each Gaussian to the image (c.f. Equation 1). Therefore, $c_i^{\\text{iso}}$ represents the contribution of the Gaussian $g_i^{\\text{iso}}$, calculated according to Equation 3 (i.e., multiplying its feature vector $\\mathbf{f}_i^{\\text{iso}}$ to the global isotropic basis vector and then applying a sigmoid activation), and similarly for $c_j^{\\text{dir}}$ being the contribution of $c_j^{\\text{dir}}$ to the pixel, after applying our proposed anisotropic model (c.f. Equation 3).\n\nWe would gladly take into account any further suggestions to make our manuscript accessible to a larger audience. E.g., we will improve our pipeline figure (Fig. 1) by adding the above-mentioned notations within, in order to help the readers understand their application.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "9KLbiJYzKy", "reviewer": "Reviewer_g5Vp", "summary": "1. The proposed direction-disentangled 3DGS (DDGS) method decomposes the radiosity contribution into isotropic and direction-dependent components, able to approximate complex anisotropic interactions without complex runtime simulations. specifically, it modeling isotropic and anisotropic contributions via distinct 3D Gaussians.\n\n2. The paper clearly mentioned that 3DGS-for-DRR solutions [30, 11] do not account for noise-inducing photon interactions (e.g., scattering) when applying the analytical methods [29, 5] to render their training data.", "questions": "1. what is the means of high-dimensional residual contribution？Is there a formal description? This concept is not explained in the full text.\n\n2. Before use the method, it need first get the 3D CT volume. The reconstruction is based on the projection obtained by the CT machine. However, there are errors and noise in the 3D CT volume (from the reconstruction of the projection noise) and also from the reconstruction errors caused by the polychromatic X-rays in medical CT machine. The paper is equivalent to reprojecting the 3D CT volume using gaussian splatting. How do the work ensure the reprojection process does not amplify this error from the 3D CT volume?", "limitations": "1. If this method is used in clinical practice, what problems will it cause without considering the time cost? For example, the appearance of artifacts, because once artifacts appear, it will seriously interfere with the doctor's clinical operation.", "rating": 4, "confidence": 4, "soundness": 2, "presentation": 2, "contribution": 2, "strengths": "1. This paper has a clear definition of the problem to be solved and a good visualization.\n2. Experiments combining 2D/3D CT Image registration were also presented.", "weaknesses": "1. Although Gaussian Splatting is widely used in natural color scenes, this paper does not provide a detailed mathematical description of the \"migrated version\" of Gaussian Splatting in combination with the physical scene of CT.  Instead, it directly use the mathematical description of Gaussian Splatting in natural color scenes (Equation (1)). In CT imaging, the contribution of a voxel to the final projection does not suffer from attenuation due to occlusion by the following voxels, but is instead Beer-Lambert law. Therefore, equation (1) no longer holds. This makes it difficult to highlight the motivation of this paper in terms of algorithm details.\n\n2.  Theoretical contributions appear to be minor. (Formula (3) is the theoretical contributions of this paper). This makes this paper more suitable for delivery to places that are more concerned with simulation applications.\n\n3. The paper needs to compare with Physics-based Monte-Carlo simulations [2, 15, 3 , 4 ]. Although these methods are time-consuming, they guarantee the quality of reprojection. The paper needs to clearly point out the difference in image quality with Physics-based Monte-Carlo simulations. The ground truth(GT) needs to select the real projection obtained by the CT machine. So the comparison table will be：\n______________________________________________________________________________________\n                         |    Physics-based Monte-Carlo simulations     |     3DGS   |  X-Gaussian  |   DDGS\n_____________________________________________________________________________________\nPSNR with GT |  \n______________________________________________________________________________________\nSSIM  with GT | \n______________________________________________________________________________________", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "### Beer-Lambert law:\nWe would like to kindly correct the reviewer. The exponential transmittance model $T(t)$ used in NeRF/GS, to describe the light attenuation as it travels through a medium from point $r(t_0)$ to $r(t)$, is **also based on Beer-Lambert law** [17, r1-4]: \n\n$T(t)=\\exp(-\\int_{t_0}^t a(r(s))\\delta s)$, \n\nwhere $r(t)=o+td$ is a 3D point at step $t$ on a ray of origin $o$ and direction $d$, and $a(p)$ is the medium's absorption/extinction (linked to its density) at point $p$. Assuming a piecewise homogeneous medium (with $a_i$ the absorption of homogeneous segment $l$ of length $\\delta_l$), this transmittance can be rewritten: \n\n$T(t)=\\exp(-\\sum_{l=1}^{l_t} a_l\\delta_l)=\\prod_{l=1}^{l_t}{(1-\\sigma_l)}$, \n\nas in our Eq. 1; with $\\sigma_l=1-\\exp(-a_l\\delta_n)$ and $l_t$ the end segment containing $t$. See [r4] for proof.\n\nX-ray attenuation from photoelectric effect (PE) follows the same model: at each step through the medium (e.g., voxel), the probability of photon extinction (i.e., averaged attenuation) is proportional to the atomic density. E.g., if a ray travels through only 2 voxels, and if it is attenuated by $a_1$=50% at step 1 then $a_2$=50% again at step 2, then its total attenuation is 75% (c.f. $(100-(100-50)^2)$%), in accordance to Beer-Lambert.\n\nHence, Eq. 1 (3DGS rendering) could be directly applied to DDR synthesis (assuming standard _neglog_ scaling of absorption values into pixel ones [12, 33]), if we were to adopt a simplified model of X-ray imaging which only considers isotropic absorption (i.e., following the Beer-Lambert law), though the term $c_j\\in\\mathbb{R}^{K}$ (view-dependent radiance) which could be omitted. This is the simplified model proposed in prior work [7], as well as concurrent work [r5] (published online after our submission).\nIn our paper, we propose to go beyond their isotropic simplification and further account for anisotropic scattering of X-rays. This is why we preserve $c_j$ and decompose it into two terms: an isotropic term $c^{\\text{iso}}$ and direction-dependent non-linear term $c^{\\text{dir}}$ (Eq. 3). Note, we also fix the dimension $K$ of these variables (= number of output channels) to 1 (monochromatic DRR) instead of 3 in usual 3DGS (RGB). We will clarify this in the paper.\n\n### Contributions w.r.t. 3DGS:\nWe would argue that while Eq. 3 is compact, it is not straightforward; nor it covers all our contributions (e.g., radiodensity-aware initialization). It may be easily overseen as it is formalized via subscripts ($i$ vs. $j$) but a key contribution is the decoupling of the isotropic and view-dependent terms, modeled by distinct Gaussians ($g^{\\text{iso}}_i$ and $g^{\\text{dir}}_j$) c.f. L163-164 and 171-177. This differs from prior 3DGS and DRR work [7, 24], using 1 set of Gaussians. Our dual-set idea may seem _unintuitive_ but demonstrates better compactness and accuracy. We believe **this insight benefits the community**.\n\nIn comparison, X-Gaussian [7] (ECCV 2024) adapts 3DGS to DRR rendering by: (a) simplifying view-dependent radiance into an isotropic term (similar to our $c^{\\text{iso}}$); and (b) replacing SfM initialization with uniform 3D space sampling (ignoring CT content). **Our contributions are comparatively significant**: (a) a 2nd anisotropic term $c^{\\text{dir}}$ with its own Gaussians $g^{\\text{dir}}$ (a more accurate physics adaptation of 3DGS to X-rays); and (b) an advanced initialization strategy considering CT data distribution; overall resulting in a **+1.04db PSNR increase** and a **10% decrease in needed Gaussians** compared to [7] over CTPelvic1K (Tab. 1).\n\n### MC/real comparison:\nPlease refer to our global response. We cannot compare MC and analytical solutions against real images but:\n- Tab. 1 already provides a relevant comparison between DRR tools (as candidates) and MC simulation (TIGRE [5], as GT). The high metrics for DDGS show that it approximates MC simulation well. \n- Similarly, the registration experiments (e.g., Tab. 7) implicitly provide insight into the similarity between real and synthetic data, as pose optimization is done by comparing DRRs to real target scans. \n- For a more direct comparison, we used our DDGS-based registration method to refine GT poses in Ljubljana data (c.f. aforementioned GT inaccuracies), then used the refined poses to render and **compare DRRs to real scans**. Results can be found in Tab. R1 and Fig. R1 (rebuttal PDF). E.g., when zooming, we notice that DiffDRR suffers from pixelization, unlike GS methods. Refined GT data will be made public. \n\n### High-dim. residual contribution:\nThank you, we will clarify. X-ray scattering is a complex phenomenon causing imaging noise (residual impact). The noise distribution is influenced by so many parameters (source direction and energy, medium properties, bounces, etc.), only a costly high-dimensional function could accurately model it (see Sec. 2.1 of [8] for details). Therefore, we only claim to approximate the residual impact of scattering.\n\n### CT noise:\nWe agree that the quality of DRRs is bound to the quality of the input CT scans. This issue affects all DRR and MC methods. E.g., MC-based tools segment CT scans into materials and use the segmented data as simulation input. If a 3D scan is noisy, so will the material segmentation and simulation results (see discussion in [33]). When our method uses simulators to get target images, it will face similar noise issues. However, our experiments show that DDGS barely introduces additional noise (highest PSNR in Tabs. 1-2). Compensating for CT noise is an interesting, under-explored research direction that could benefit all DRR methods; but, we believe, beyond the scope of our study.\n\n### Clinical usage:\nWhen clinicians use DRRs for quick visualization, they know the tools' limitations. E.g., DRRs are typically not used for diagnosis. We agree with the reviewer that image quality is still important, hence our experiments showing DDGS accuracy over similar DRR works.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:
We are very grateful to the reviewers `Zg8U`, `YFhr`, `ZGY5`, and `g5Vp` for the constructive feedback, as well as the recognition of our paper's strengths, such as its novelty with regard to existing analytical DRR renderers, its clarity, and its conclusive evaluation on downstream tasks.

In this global response, we further highlight the strengths of our work shared by reviewers and address the only concern brought by more than one reviewer. Other individual questions are answered in our per-reviewer responses, and we are looking forward to further discussing with reviewers.

## Shared Strengths
### 1. Technical Novelty
Reviewers `Zg8U` and `YFhr` agree that our approach is "innovative" [`Zg8U`] / "novel" [`YFhr`], and "**significant step forward in the field of medical imaging**" [`Zg8U`].

Whereas "existing real-time GS-based methods ignore complex X-ray interactions" [`YFhr`], we provide two key contributions, i.e., "introducing the disentanglement of isotropic and direction-dependent components and initialization strategy that accounts for the nature of CT data" [`Zg8U`, `YFhr`, `ZGY5`]. 

### 2. Significant Empirical Improvement
Reviewers positively responded to our quantitative evaluations and agree on the "effectiveness" [`Zg8U`, `YFhr`] / "excellence" [`ZGY5`] of our algorithm. 

Not only does DDGS "**substantially improve PSNR and SSIM** scores [...] over existing methods" [`Zg8U`], but "experiments **combining 2D/3D CT image registration** were also presented" [`g5Vp`]. Reviewer `YFhr` deemed our experiments against "multiple SOTA methods" and "on 3 different datasets" to be "extensive".

### 3. Clear Motivation and Methodology
All four reviewers agree that our motivation / definition of the problem is **clear** [`Zg8U`, `YFhr`, `ZGY5`, `g5Vp`]. Except for `ZGY5`, all reviewers also praised the structure/comprehensiveness of our methodological framework.

Finally, they also agree that we "adequately discussed [our] limitations".

## Shared Question - Comparison to MC Data
Overall, the questions and concerns brought by the reviewers have little overlap, except for the positioning of our work w.r.t. Monte-Carlo simulators, asked by `g5Vp` and implicitly `Zg8U`. We partly clarify below and further answer each reviewer in their respective response.

### MC Applications
We shall better highlight the fundamental differences between Monte-Carlo simulators and analytical DRR renderers, not only in terms of theoretical background but also in terms of target applications/metrics.

DRRs are used either to provide quick visualization to clinicians (e.g., for pre-operative planning or intra-operative guidance) or as an intermediary modality for automation algorithms (e.g., to enable CT/X-ray registration, c.f. our evaluation). Analytical methods have thus been proposed to generate such projections from CT data, navigating the trade-off between speed and photorealism.

On the other hand, MC simulators tackle broader applications (dosimetry validation, photon tracking, etc.) and are usually applied to virtual scenes (e.g., digital twins of scanners, phantoms, geometric primitives, etc.). While some provide the interface to import real CT scans into their virtual scene, the scans have to be manually preprocessed (structure segmentation and material assignment), a tedious and error-prone procedure. Combined with their extremely slow nature (c.f. per-photon simulation of complex physics effects), this makes them ill-suited for online DRR rendering. I.e., their application to DRR rendering is only as a costly source of GT images, not as actual solutions to be compared with.

### Standard Evaluation of DRR Methods
The main goals of DRRs are to provide quick visualization to clinicians (key metrics = speed, visibility) and to integrate larger imaging applications (key metrics: speed, feature-level similarity with real data).

Therefore, prior DRR papers (traditional [11, 29, 30] or GS-based [7, 24]) mostly evaluate on downstream tasks (e.g., pose registration). A few evaluate the image quality compared to other DRR tools ([11] compares to Siddon's and Plastimatch [29]; [7] to other NVS baselines). We found only one [12] that provides a qualitative comparison to some real scans. Quantitative comparison to real data is challenging. Not all imaging parameters are usually provided or accurate enough to create matching DRRs (w.r.t. intensity, CT pose, etc.). It is especially hard to render DRRs aligned with real scans with existing MC tools [2-4, 15]; which is likely why **no DRR paper has performed such evaluation**. Their interfaces and custom coord. conventions are not compatible with pose annotations in public CT/X-ray datasets [13, 25], and their documentation/support is lacking (we tried reaching their authors). Bridging the convention gap between MC and analytical communities would greatly benefit our domain and could be the focus of our next effort. 
Note another benefit of DDGS: if real posed scans are available, it could be finetuned on them for higher realism, which is not possible with traditional DRR tools and MC simulators. 

### Additional Results
Please refer to our responses to `g5Vp` and `Zg8U` to see **additional results** that we nevertheless successfully gathered.

----
### References mentioned in our responses:
[r1] Mildenhall et al. Nerf: Representing scenes as neural radiance fields for view synthesis. ACM-Comm, 2021.

[r2] Vicini et al. A non-exponential transmittance model for volumetric scene representations. TOG, 2021.

[r3] Garbin et al. Fastnerf: High-fidelity neural rendering at 200fps. ICCV, 2021.

[r4] Tagliasacchi et al. Volume rendering digest (for nerf). arXiv, 2022.

[r5] Zha et al. R2-Gaussian: Rectifying Radiative Gaussian Splatting for Tomographic Reconstruction. arXiv, 2024.

[r6] Fedorov et al. 3D Slicer as an image computing platform for the Quantitative Imaging Network. Magnetic resonance imaging, 2012.

Meta-review: The manuscript introduces the Direction-Disentangled Gaussian Splatting (DDGS-CT) method, aimed at improving the balance between realistic X-ray simulation and efficient DRR generation using 3D Gaussian Splatting (3DGS). The method distinguishes itself by decomposing the radiosity into isotropic and direction-dependent components, which approximates complex anisotropic interactions without runtime simulation. The proposed technique demonstrates significant improvements in PSNR and SSIM for DRR renderings of multiple organs, suggesting superior performance compared to existing methods.

Strengths : 
The DDGS-CT method introduces a novel technique for balancing efficiency and realism in DRR rendering, which is a significant step forward in medical imaging.
The method models isotropic and anisotropic contributions via distinct 3D Gaussians to handle complex X-ray interactions like Compton scattering. 
It demonstrates substantial improvements in PSNR and SSIM scores, validating its effectiveness over existing methods.

Weaknesses :
The authors have adequately discussed their limitations, but some concerns remain about the robustness and reliability of the method in clinical practice, particularly regarding artifacts and the amplification of errors.

Recommendations:
The authors could discuss the results a little more to show the effectiveness of their method in terms of benefits which sometimes seem marginal. For example, the notion of noise or imprecision in the initial segmentation concerns all methods in any case.

The authors have responded fully and honestly to the reviewers' many comments. They have provided additional information, tables and figures to improve the presentation of their results.
Decision: Accept (poster)