# Multi-stage CNN for fast registration of 3D preoperative CTs to 2D intraoperative X-rays

Federica Facente, Benjamin Billot, Vivek Gopalakrishnan, Manasi Kattel, Wen Wei, Polina Golland, Hervé Delingette, Nicholas Ayache, Pierre Berthet-Rayne

► To cite this version:

Federica Facente, Benjamin Billot, Vivek Gopalakrishnan, Manasi Kattel, Wen Wei, et al. Multi-stage CNN for fast registration of 3D preoperative CTs to 2D intraoperative X-rays. MICCAI 2025 - COLlaborative Intelligence and Autonomy in Image-guided Surgery (COLAS), Sep 2025, Daejon, South Korea. hal-05218972

HAL Id: hal-05218972

https://hal.science/hal-05218972v1

Submitted on 22 Aug 2025

HAL is a multi-disciplinary open access
archive for the deposit and dissemination of sci-
entific research documents, whether they are pub-
lished or not. The documents may come from
teaching and research institutions in France or
abroad, or from public or private research centers.

L'archive ouverte pluridisciplinaire HAL, est
destinée au dépôt et à la diffusion de documents
scientifiques de niveau recherche, publiés ou non,
émanant des établissements d'enseignement et de
recherche français ou étrangers, des laboratoires
publics ou privés.

<figure><img src="image_1.png" /></figure>

Distributed under a Creative Commons Attribution 4.0 International License
Multi-stage CNN for fast registration of 3D
preoperative CTs to 2D intraoperative X-rays

Federica Facente¹, Benjamin Billot¹, Vivek Gopalakrishnan³, Manasi Kattel¹,
Wen Wei², Polina Golland³, Hervé Delingette¹*, Nicholas Ayache¹, and Pierre
Berthet-Rayne²

1 Université Côte d'Azur, Inria, Epione Team, Sophia Antipolis, France

² Caranx Medical, Nice, France

³ CSAIL, MIT, Cambridge, United States
federica.facente@inria.fr

**Abstract.** Minimally invasive interventions often rely on live 2D X-rays for image guidance. Yet, anatomical localization and procedural accuracy can be enhanced by spatial alignment of these intraoperative X-rays with 3D preoperative computed tomographies (CTs). This 3D/2D registration problem is typically formulated as pose estimation of the X-ray source relatively to the CT, which is done by simulating synthetic X-rays from 2D projections of CT volumes. However, the optimization-based refinement used by the state-of-the-art deep learning approach takes several seconds, thus exceeding the allowed time budget in live image guidance. In this paper, we propose LXPose (Live X-ray Pose estimation), a self-supervised multi-stage 3D/2D registration framework for real-time image guidance. LXpose removes the dependency on optimization and leverages a two-stage CNN trained with a projection loss to ensure high accuracy and computational efficiency. Moreover, we apply extensive data augmentation to mitigate the domain gap between simulated and real X-rays. Overall, LXPose yields comparable 2D registration error to the state-of-the-art method, while reducing inference time to 20ms, which demonstrates the potential of LXPose for real-time clinical deployment. Our code is available at https://github.com/fedefacente/LXPose.git.

**Keywords:** Pose estimation · Multi-stage CNN · Image guidance

# 1 Introduction

Minimally invasive interventions are often guided using real-time X-rays, which are acquired with a mobile C-arm device. However, X-ray imaging inherently lacks depth information since it is a 2D projection of 3D anatomy. For this reason, intervention planning is often performed on preoperative 3D computed tomographies (CTs) of high resolution and superior soft tissue contrast. CT-derived preoperative information is then mentally projected by the clinician onto the intraoperative X-rays, which increases cognitive overload and risk of errors [36,

* H. Delingette, N. Ayache, and P. Berthet-Rayne contributed equally to this work.
30]. A solution to this problem is to automatically register the CT volumes onto the live X-ray images, thus effortlessly providing the clinician with preoperative information. Nonetheless, CT to X-ray registration remains a major challenge due to the complexity of the 3D/2D mapping.

Early methods are based on optimization, and use the fact that X-rays and CTs share the same imaging physics. Indeed, X-rays can be realistically approximated as 2D projections of 3D CTs. These projections, known as digitally reconstructed radiographs (DRR), are simulated by virtually casting rays from the X-ray source (i.e., the C-arm) and computing the attenuation along ray propagation through the CT [24, 21]. As a result, optimization strategies formulate the CT/X-ray registration task as pose estimation of the C-arm relatively to the CT volume, so that preoperative information can easily be projected onto intraoperative X-rays. Specifically, pose prediction is performed by optimizing a similarity metric between pose-dependent DRRs and the real X-rays [13, 1, 25]. While a more recent simulation framework (DeepDRR [33]) improves the accuracy of optimization methods [15], these remain sensitive to local minima.

Modern CT/X-ray registration methods mostly rely on deep learning. A first class of methods performs C-arm pose estimation by training supervised X-ray landmark detection networks in either cross-subject settings (for generalizability) [2, 27, 34] or subject-specific scenarios (for enhanced accuracy) [14, 6, 12]. The detected landmarks are then matched to 3D CT keypoints via Perspective-n-Point (PnP) [16] for final pose estimation. However, these methods require labor-intensive manual annotations of matched 3D/2D landmarks. Recent approaches propose to reduce this dependency by regressing 2D projections of dense surface coordinates obtained from CT segmentations [31, 18], but such methods remain sensitive to occlusions by surgical tools. Another class of methods involves directly regressing the C-arm pose from input X-rays using patient-specific models trained on simulated DRRs [23, 3, 26] possibly along with real X-rays paired with cumbersome ground truth poses [38, 8]. More generally, predicting accurate poses in only one forward pass remains challenging. That is why, state-of-the-art methods propose to treat the poses regressed by networks as initialization for optimization-based refinement [9, 10, 37]. Importantly, these methods introduce differentiable variants of DeepDRR, namely DiffDRR [11] and diffDeepDRR [37], to enable gradient-based optimization. However, this optimization step takes several seconds, which far exceeds the 40ms constraint imposed by the maximum acquisition rate of X-ray systems used for real-time image guidance. Finally, all previous methods suffer from the substantial domain gap between synthetic DRRs and real X-rays [32, 4].

Here, we propose LXPose (Live X-ray Pose estimation), a novel real-time multi-stage patient-specific convolutional neural network (CNN) for fast and accurate pose prediction. Specifically, we (i) employ a two-stage CNN that progressively refines pose estimates while harnessing the speed of a single forward pass for real-time deployment; (ii) minimize a mean Projection Distance (mPD) training loss to directly optimize alignment in the space of 2D X-rays used during intervention; (iii) precisely quantify the domain gap between real X-rays
<figure><img src="image_2.png" /><figcaption>**Fig. 1.** (A) Method overview. LXPose performs CT/X-ray registration by progressively estimating the pose of the X-ray source from which 2D projections of the CT (i.e., DRRs) best align with the input X-ray. (B) LXPose architecture. A first CNN regresses the pose parameters from a 2D input image. This initial pose is then used to simulate a corresponding DRR. Finally, the latter is used by a second CNN along with the input image to predict a corrective term for final pose estimation.</figcaption></figure>

and DRRs, and partly address it with specific data augmentations. Overall, our method achieves comparable 2D projection error as the state-of-the-art methods, while enabling real-time integration of CT-based information during intervention by considerably reducing inference time to 20ms.

# 2 Methods

Our goal is to register 3D preoperative CT volumes to 2D intraoperative X-rays. This is achieved by training a multi-stage CNN for progressive pose estimation of the C-arm in the coordinate system of the CT scanner. In line with the literature, LXPose is trained using synthetic DRRs simulated by casting virtual rays through the preoperative CT [9, 37]. Our method is illustrated in Figure 1.

## 2.1 X-ray projective geometry

The C-arm is a mobile system with six degrees of freedom (6DoF). Its pose is defined by a translation $t = (t_x, t_y, t_z) \in \mathbb{R}^3$ controlling the in-plane position $(t_x, t_y)$ and magnification $t_z$, as well as by a rotation $r = (\alpha, \beta, \gamma) \in \mathbb{R}^3$, where $\alpha$ and $\beta$ are the orbital and secondary angles in the plane of the C-arm gantry, and $\gamma$ is the angle around the normal of the detector.

Building on previous works, we model the geometric projection of CT volumes to 2D DRRs using the pinhole camera model [16], which describes how 3D points are projected onto a 2D image. This model relies on a $\mathbb{R}^{3 \times 4}$ projection matrix $\Pi$ that maps 3D world points in homogeneous coordinates $(x, y, z, 1)$ onto 2D pixels homogeneous coordinates $(u, v, 1)$. This projection matrix $\Pi$ can be decomposed into extrinsic and intrinsic matrices. The former, which is at the core of this paper, is the $\mathbb{R}^{4 \times 4}$ matrix corresponding to the rigid transform that converts world coordinates (here the CT scanner coordinates) to the camera coordinate system. Thus, the extrinsic matrix is equivalent to the pose $P$ of the C-arm, and is fully described by $[r, t]$. Then, the intrinsic component is a $\mathbb{R}^{3 \times 4}$ matrix that
transforms points from the camera coordinate system to pixel locations in image
space (i.e., the 2D X-ray), and depends on the camera properties: focal length
*f*, pixel size, and image center.

Overall, given a calibrated C-arm system with known intrinsic properties,
the 2D projection X^2D of a 3D point X^3D in homogeneous coordinates is fully
determined by the pose P of the C-arm: X^2D = Π_P(X^3D).

## 2.2 Synthetic X-ray simulation

While not being the focus of this paper, we summarize here the principles of DRR simulation, and refer the reader to [11, 37] for more detailed descriptions. Briefly, DRR simulation frameworks model the attenuation undergone by X-rays when travelling through tissues [24]. This model builds on the fact that CTs are acquired using the same imaging physics as X-rays [21]. Specifically, DRRs are simulated by casting rays from a given position of the C-arm through 3D CTs. Pixel values of the DRR are then obtained by simulating the intensity of the outgoing rays, which linearly depends on attenuation coefficients defined by the density of the traversed tissues [33].

## 2.3 Multi-stage CNN

LXPose is a two-stage CNN framework that takes a 2D X-ray as input and predicts the corresponding 6DoF C-arm pose in the space of the 3D CT. This is achieved by using a first network CNN1 to regress an initial estimate of $[r, t] \in \mathbb{R}^6$ (Figure 1.B). Here, *r* is modelled with an axis-angle parametrization, which slightly improved results in preliminary experiments compared to Euler angles used in previous works [9]. A corresponding $\mathbb{R}^{4 \times 4}$ representation of this initial predicted pose $P_{\text{init}}$ is then constructed using the Rodrigues Formula [16].

To enhance robustness and accuracy, a second network CNN2 refines the initial prediction. Specifically, CNN2 receives both the input X-ray and a DRR simulated from the first pose estimate (initial DRR in Figure 1.B), and predicts a corrective transform $P_\Delta$, which is composed to $P_{\text{init}}$ to obtain a final pose $P_{\text{final}} = P_\Delta \cdot P_{\text{initial}}$. Overall, CNN2 replaces the time-consuming optimization used in state-of-the-art methods [9, 10, 37], thus providing a computationally efficient alternative suitable for real-time clinical use.

## 2.4 Learning

Building on the CT/X-ray deep learning registration literature, we train LX-Pose using simulated DRRs, which suppresses the need for ground truth poses or matched 3D/2D landmarks annotated on real data. Here, we propose to use an anatomical supervision that relies only on a set of *N* CT-derived 3D anatomical landmarks $\{X_n^{3D}\}_{1 \le n \le N}$. Specifically, we optimize the mean Projection Distance
(mPD) loss, defined as the average Euclidean distance between the 2D projections of the 3D landmarks using the ground truth and predicted poses:

$$ \mathcal{L}_{\text{mPD}} = \frac{1}{N} \sum_{n=1}^{N} \| \Pi_{P_{GT}}(X_n^{3D}) - \Pi_{P_{\text{final}}}(X_n^{3D}) \|_2, \quad (1) $$

where $P_{GT}$ is the ground truth C-arm pose. For clinical interpretability, the mPD is reported in millimeters using the known pixel size of the X-ray system. Moreover, during the loss computation, we discard landmarks whose 2D ground truth falls outside the field of view of the reference DRR, which slightly improved results in preliminary experiments by allowing the network to focus only on visible projected points. More generally, we emphasize that, as opposed to state-of-the-art methods that employ geometric losses directly defined on the regressed poses [9, 37], our network is trained by optimization in the 2D projection space, which better reflects the clinical scenario of image guidance performed on the 2D intraoperative X-rays.

Furthermore, we take advantage of using a differentiable DRR rendering process [11] to complement this geometric loss with an intensity-based loss that compares the input DRR with the synthetic DRRs simulated using the initial and final estimated poses. Inspired by previous works, we adopt here a multiscale normalized cross-correlation (mNCC) loss $\mathcal{L}_{\text{mNCC}}$, due to its effectiveness in capturing both structural and intensity correspondences [10].

In summary, LXPose is trained end-to-end using the following loss function:

$$ \mathcal{L} = (\lambda \mathcal{L}_{\text{mPD}} + \mathcal{L}_{\text{mNCC}})_1 + (\lambda \mathcal{L}_{\text{mPD}} + \mathcal{L}_{\text{mNCC}})_2, \quad (2) $$

where the first and second terms are computed on the outputs of CNN1 and CNN2, respectively, and $\lambda$ balances the effect of $\mathcal{L}_{\text{mPD}}$ and $\mathcal{L}_{\text{mNCC}}$.

## 2.5 DRR/X-ray domain adaptation with data augmentation

To address the domain shift between the training DRRs and the real X-rays seen at test-time, we apply aggressive data augmentation: Gaussian noise, Gaussian blur, sharpening, and gamma transforms. In addition to these augmentations commonly used in the CT/X-ray registration literature, we introduce two new X-ray-specific augmentations to further reduce the domain gap. First, we include salt-and-pepper noise (SP) to simulate defective sensors in the X-ray detector. Then, we leverage plasma intensity variations [28] to render inhomogeneities in X-ray contrasts due to beam hardening and the anode heel effect [5]. More precisely, plasma augmentations use the diamond-square algorithm [7] to generate fractal maps with multi-scale, spatially varying intensities. These maps are then applied to the training DRRs with element-wise multiplication.

## 2.6 Implementation details

**DRR simulation** Training DRRs are simulated on the fly using DiffDRR [11]. Here, we place the source at the origin and the detector at $(0, 0, f)$. We reorient
the depth axis to align with the CT's posterior-anterior axis, and translate the source and detector along the z-axis by 750mm to place them around the subject. DRRs are then simulated by sampling C-arm pose parameters from normal centered distributions with the following standard deviations: 15° ($\alpha, \beta$), 5° ($\gamma$), 50mm ($t_x, t_y$), 100mm ($t_z$). As in previous works [9, 37], we simulate DRRs at reduced size (256×256) for memory usage.

**Architecture and training** Each CNN consists in a ResNet-18 [17] followed by a fully connected layer. Based on validation scores, we set $\lambda = 10^{-1}$. Training is conducted for 2000 epochs with a batch size of 16 using Adam [20] with an initial learning rate of $10^{-3}$ and a warmup cosine scheduler [35]. Our network is implemented using PyTorch [29]. Training takes 20 hours on a 80Gb Nvidia A100 GPU, and inference runs in 20ms on a 12Gb Nvidia 1080Ti GPU.

# 3 Experiments and results

## 3.1 Dataset and preprocessing

We evaluate our pipeline on the DeepFluoro dataset [14], an open-source collection of six cadaveric subjects, each comprising a CT volume as well as an X-ray series between 24 and 111 images (366 in total). The C-arm is fully calibrated with known intrinsic parameters, and every X-ray is provided with ground truth extrinsic camera parameters, only used for testing purposes. Moreover, CTs are available with 14 manual anatomical 3D landmarks and the coordinates of their corresponding 2D projections in the test X-rays. Real X-rays are held-out for testing, whereas training and validation are performed on patient-specific synthetic DRRs to relax supervision requirements and to simulate clinical settings where networks are trained preoperatively without available X-rays.

Here, we adopt the X-ray absorption convention to match the DiffDRR formulation. Hence, we convert raw intraoperative attenuation images $I$ into absorption images $I_{\mu} = \log[\max(I)] - \log I$, where the brightest pixels $\max(I)$ correspond to rays that travel through air only. Finally, we crop all images to remove the collimator effect, and resample them to 256×256.

## 3.2 Competing methods

The proposed LXPose approach is compared against 3 baseline methods: PoseNet is a network trained on real X-rays to regress C-arm poses from ground truth poses [19]. Since the associated code is not published, we re-implement PoseNet and train it on real X-rays using leave-one-subject-out cross-validation. xvr-regressor is the regression CNN used in xvr [9], and differs from LXPose by: not being multi-stage, replacing $L_{mPD}$ by a geodesic loss, regressing Euler angles instead of axis-angles, and not using the newly introduced augmentations. xvr is the state-of-the-art method proposed by [9] with regression and optimization steps. We test it for reference only as it far exceeds the 40ms time limit.

For fair comparison, all networks are trained using the same architecture (i.e., ResNet-18), preprocessing, and generic augmentations as LXPose.
<figure><img src="image_3.png" /><figcaption>**Fig. 2.** CT/X-ray registration results for all methods visualized as: the target X-ray with 3D-derived landmarks projected from the ground truth pose, DRR with the landmarks projected from the predicted pose, and intensity differences between the two.</figcaption></figure>

<table><caption>**Table 1.** Mean (standard deviation) scores for all methods. Best scores are in bold and stars mark statistical significance (Bonferroni-corrected Wilcoxon signed-rank test, 1% level). All differences between LXPose and xvr are also significant (same test).</caption><thead><tr><th>Metric</th></tr></thead><tbody><tr><td>mPD (mm)</td></tr><tr><td>mTRE (mm)</td></tr><tr><td>α (°)</td></tr><tr><td>β (°)</td></tr><tr><td>γ (°)</td></tr><tr><td>t<sub>x</sub> (mm)</td></tr><tr><td>t<sub>y</sub> (mm)</td></tr><tr><td>t<sub>z</sub> (mm)</td></tr><tr><td>time (s)</td></tr></tbody></table>

<table><thead><tr><th></th><th>PoseNet [19]</th><th>xvr-regressor [9]</th><th>LXPose (ours)</th></tr></thead><tbody><tr><td rowspan="2">40ms time constraint</td><td>77.70 (22.58)</td><td>4.07 (2.06)</td><td><strong>1.25 (1.04)*</strong></td></tr><tr><td>187.18 (22.58)</td><td>32.41 (18.29)</td><td><strong>13.52 (10.48)*</strong></td></tr><tr><td rowspan="2">No time constraint</td><td>10.42 (7.31)</td><td>1.35 (1.18)</td><td><strong>0.46 (0.46)*</strong></td></tr><tr><td>5.90 (4.12)</td><td>1.55 (1.30)</td><td><strong>0.73 (0.78)*</strong></td></tr><tr><td rowspan="2">40ms time constraint</td><td>7.39 (3.30)</td><td>0.98 (0.73)</td><td><strong>0.33 (0.35)*</strong></td></tr><tr><td>15.90 (11.81)</td><td>1.20 (0.93)</td><td><strong>0.33 (0.37)*</strong></td></tr><tr><td rowspan="2">No time constraint</td><td>48.00 (16.37)</td><td>1.03 (0.78)</td><td><strong>0.33 (0.27)*</strong></td></tr><tr><td>62.91 (37.56)</td><td>12.67 (9.56)</td><td><strong>4.73 (5.07)*</strong></td></tr><tr><td>time (s)</td><td>0.007 (0.032)</td><td>0.007 (0.032)</td><td>0.020 (0.033)</td></tr></tbody></table>

<table><thead><tr><th></th><th>xvr [9]</th></tr></thead><tbody><tr><td rowspan="2">40ms time constraint</td><td>0.62 (1.24)</td></tr><tr><td>4.74 (11.39)</td></tr><tr><td rowspan="2">No time constraint</td><td>0.21 (0.45)</td></tr><tr><td>0.25 (0.75)</td></tr><tr><td rowspan="2">40ms time constraint</td><td>0.08 (0.16)</td></tr><tr><td>0.14 (0.24)</td></tr><tr><td rowspan="2">No time constraint</td><td>0.14 (0.31)</td></tr><tr><td>1.86 (3.73)</td></tr><tr><td>time (s)</td><td>7.777 (2.094)</td></tr></tbody></table>

## 3.3 Results

**Registration performances** We evaluate our results using mean distances between the registered and ground truth landmarks, both in 3D with mTRE [22] and in 2D (i.e., after projection) with mPD. Note that mTRE is reported for completeness since mPD is more insightful for image guidance in the 2D intraoperative X-ray space. We also assess mean absolute errors of the estimated pose parameters α, β, γ, t<sub>x</sub>, t<sub>y</sub>, t<sub>z</sub>. Finally, we report the inference time per image, which is critical for real-time image guidance.

While xvr yields the best scores for all metrics (Table 1), inference requires more than 7 seconds due to the optimization step, making it unsuitable for real-time use. In comparison, LXPose offers a favorable trade-off: although xvr obtains slightly better results (difference of 0.6mm in mPD), LXPose remains highly accurate (see Figure 2) while yielding a mean inference time of just 20ms, thus complying with the 40ms time constraint for intraoperative deployment. In-
<table><caption>**Table 2.** Mean (standard deviation) scores for all ablations. Best scores are in bold and * marks stat. significance (Bonferroni-corrected Wilcoxon signed-rank test, 1% level).</caption><thead><tr><th rowspan="2">Ablations</th><th colspan="2">real test X-rays</th><th colspan="2">synthetic test DRRs</th></tr><tr><th>mPD (mm)</th><th>mTRE (mm)</th><th>mPD (mm)</th><th>mTRE (mm)</th></tr></thead><tbody><tr><td>LXPose (ours)</td><td><strong>1.25 (1.04)*</strong></td><td>13.51 (10.47)</td><td><strong>0.55 (0.66)*</strong></td><td><strong>3.02 (2.20)*</strong></td></tr><tr><td>LXPose ⊖ CNN2</td><td>1.75 (1.39)</td><td>17.81 (13.83)</td><td>1.33 (1.54)</td><td>7.80 (5.38)</td></tr><tr><td>LXPose, sequential training</td><td>1.75 (1.35)</td><td>17.51 (13.35)</td><td>1.45 (1.56)</td><td>7.91 (4.90)</td></tr><tr><td>LXPose, Euler angles</td><td>2.48 (1.85)</td><td>19.72 (17.08)</td><td>1.99 (2.38)</td><td>11.8 (10.00)</td></tr><tr><td>LXPose, geodesic loss</td><td>2.60 (3.60)</td><td><strong>11.73 (10.82)*</strong></td><td>1.82 (1.65)</td><td>3.08 (2.36)</td></tr><tr><td>LXPose ⊖ plasma and SP aug.</td><td>4.01 (1.82)</td><td>45.61 (19.75)</td><td>0.75 (0.65)</td><td>5.14 (2.96)</td></tr><tr><td>LXPose ⊖ all (= xvr-regressor)</td><td>4.07 (2.06)</td><td>32.41 (18.29)</td><td>2.19 (2.76)</td><td>9.43 (12.31)</td></tr></tbody></table>

terestingly, the gap in mTRE is substantially larger than mPD. This discrepancy is mainly driven by errors along the 3D depth axis parameter $t_z$, which primarily affect magnification but have a limited impact once projected onto the 2D image plane. Finally, LXPose significantly outperforms every baseline that satisfy the 40ms time constraint, with, for example, gaps of 2.82mm and 76.45mm in mPD for xvr-regressor and PoseNet, respectively. These results highlight the quality of our predictions in the allowed time budget for real-time guidance.

**Ablation study** We now conduct ablations to study the impact of each component of LXPose (Table 2). First, removing the network CNN2 leads to a significant performance drop, thus highlighting the role of our multi-stage formulation for progressive refinement. Then, we observe that training end-to-end improves accuracy by enabling CNN1 and CNN2 to learn synergies, as opposed to sequential training. Further, regressing axis-angle representation of $r$ yields more robust estimates than using Euler angles. Finally, while the geodesic loss employed in [9] yields narrowly better mTRE scores (gap of 1.7mm), our loss shows better mPD scores (1.3mm gap), which is the metric of choice for 2D image guidance.

**Domain gap quantification** In this last experiment, we precisely quantify the domain gap between the target X-ray domain and the training DRR domain. To this end, we leverage the ground truth C-arm pose associated with each real X-ray to simulate a corresponding test DRR. As expected, all networks perform better on the DRR source domain (0.92mm mPD difference with real X-rays across methods). Interestingly, ablating the newly introduced plasma and SP augmentations leads to a far greater gap (3.26mm). This ablation also mainly explains our improvements over xvr-regressor (i.e., fully ablated LXPose), thus showing the crucial role of our augmentations in tackling the domain gap.

## 4 Conclusion

We have presented LXPose, an end-to-end deep learning framework for real-time 3D/2D registration of preoperative CTs to intraoperative X-rays. Remarkably,
LXPose almost reaches the projection accuracy of the state-of-the-art method, but runs considerably faster to enable real-time image-guidance. This has been achieved by leveraging deep learning for speed, a multi-stage CNN for progressive pose estimation, a 2D loss for accurate projection in the intraoperative image space, and an aggressive augmentation pipeline for enhanced robustness on real data. Future work will first assess our method on additional datasets (e.g., interventional cardiology), possibly with real-time deployment. To this end, we will also aim to further bridge the DRR/X-ray domain gap to improve robustness on real X-rays. Finally, we will extend our method to incorporate pre-training on multi-patient data to increase generalization [9]. By providing precise real-time guidance, LXPose has the potential to reduce procedure durations, alleviate the cognitive burden of clinicians, and improve interventional outcomes.

**Acknowledgments.** This work has been supported by the French government, through the 3IA Cote d'Azur Investments in the project managed by the National Research Agency (ANR) with the reference number ANR-23-IACL-0001. Further support was provided with computer resources by GENCI at CINES/IDRIS thanks to the grant 2025- A0180315107 on the supercomputer Jean Zay's V100/A100 Partition.

**Disclosure of Interests.** The authors have no competing interests to declare.

## References

1. Berger, M., Müller, K., Aichert, A., Unberath, M., Thies, J., Choi, J., Fahrig, R., Maier, A.: Marker-free motion correction in weight-bearing cone-beam CT of the knee joint. Medical Physics 43(3), 1235-1248 (2016)

2. Bier, B., Unberath, M., Zaech, J.N., Fotouhi, J., Armand, M., Osgood, G., Navab, N., Maier, A.: X-ray-transform invariant anatomical landmark detection for pelvic trauma surgery. In: MICCAI. pp. 55-63 (2018)

3. Bui, M., Albarqouni, S., Schrapp, M., Navab, N., Ilic, S.: X-Ray PoseNet: 6 DoF pose estimation for mobile X-ray devices. In: WACV. pp. 1036-1044 (2017)

4. Chen, J., Liu, Y., Wei, S., Bian, Z., Subramanian, S., Carass, A., Prince, J., Du, Y.: A survey on deep learning in medical image registration: New technologies, uncertainty, evaluation metrics, and beyond. Medical Image Analysis 100 (2025)

5. Chou, M.C.: Evaluation of non-uniform image quality caused by anode heel effect in digital radiography using mutual information. Entropy 23, 525 (2021)

6. Esteban, J., Grimm, M., Unberath, M., Zahnd, G., Navab, N.: Towards fully automatic X-ray to CT registration. In: MICCAI. pp. 631-639 (2019)

7. Fournier, A., Fussell, D., Carpenter, L.: Computer rendering of stochastic models. Communications of the ACM 25(6), 371-384 (1982)

8. Geng, H., Xiao, D., Yang, S., Fan, J., Fu, T., Lin, Y., Bai, Y., Ai, D., Song, H., Wang, Y., Duan, F., Yang, J.: CT2X-IRA: CT to X-ray image registration agent using domain-cross multi-scale-stride deep reinforcement learning. Physics in Medicine & Biology 68(17) (2023)

9. Gopalakrishnan, V., Dey, N., Chlorogiannis, D., Abumoussa, A., Larson, A., Orbach, D., Frisken, S., Golland, P.: Rapid patient-specific neural networks for intraoperative X-ray to volume registration. arXiv:2503.16309 (2025)
10. Gopalakrishnan, V., Dey, N., Golland, P.: Intraoperative 2D/3D image registration via differentiable X-ray rendering. CVPR pp. 11662–11672 (2024)

11. Gopalakrishnan, V., Golland, P.: Fast auto-differentiable digitally reconstructed radiographs for solving inverse problems in intraoperative imaging. In: Workshop on Clinical Image-Based Procedures (2022)

12. Grimm, M., Esteban, J., Unberath, M., Navab, N.: Pose-dependent weights and domain randomization for fully automatic X-ray to CT registration. IEEE transactions on medical imaging 40(9), 2221–2232 (2021)

13. Groher, M.: 2D-3D Registration of Vascular Images: Towards 3D-guided catheter interventions. Ph.D. thesis, Technische Universität München (2008)

14. Grupp, R., Unberath, M., Gao, C., Hegeman, R., Murphy, R., Alexander, C., Otake, Y., McArthur, B., Armand, M., Taylor, R.: Automatic annotation of hip anatomy in fluoroscopy for robust and efficient 2D/3D registration. International Journal of Computer Assisted Radiology and Surgery 15(5), 759–769 (2020)

15. Gu, W., Gao, C., Grupp, R., Fotouhi, J., Unberath, M.: Extended capture range of rigid 2D/3D registration by estimating riemannian pose gradients. In: International Workshop on Machine Learning in Medical Imaging. pp. 281–291 (2020)

16. Hartley, R.: Multiple view geometry in computer vision, vol. 665. Cambridge university press (2003)

17. He, K., Zhang, X., Ren, S., Sun, J.: Deep residual learning for image recognition. In: CVPR. pp. 770–778 (2016)

18. Jaganathan, S., Kukla, M., Wang, J., Shetty, K., Maier, A.: Self-supervised 2D/3D registration for X-ray to CT image fusion. In: WACV. pp. 2788–2798 (2023)

19. Kendall, A., Grimes, M., Cipolla, R.: PoseNet: A convolutional network for real-time 6-DOF camera relocalization. In: ICCV. pp. 2938–2946 (2015)

20. Kingma, D., Ba, J.: Adam: A Method for Stochastic Optimization. arXiv:1412.6980 (2014)

21. Knaan, D., Joskowicz, L.: Effective intensity-based 2D/3D rigid registration between fluoroscopic X-ray and CT. In: MICCAI. pp. 351–358 (2003)

22. Van de Kraats, E., Penney, G., Tomaževič, D., van Walsum, T., Niessen, W.: Standardized evaluation of 2D-3D registration. In: MICCAI. pp. 574–581 (2004)

23. Lee, B., Sinha, A., Varble, N., Pritchard, W., Karanian, J., Wood, B., Bydlon, T.: Breathing-compensated neural networks for real time C-arm pose estimation in lung CT-fluoroscopy registration. In: ISBI (2022)

24. Lemieux, L., Jagoe, R., Fish, D., Kitchen, N., Thomas, D.: A patient-to-computed-tomography image registration method based on digitally reconstructed radiographs. Medical physics 21, 1749–1760 (1994)

25. Meng, C., Wang, Q., Guan, S., Sun, K., Liu, B.: 2D-3D registration with weighted local mutual information in vascular interventions. IEEE Access 7, 629–638 (2019)

26. Miao, S., Wang, Z.J., Liao, R.: A CNN regression approach for real-time 2D/3D registration. IEEE transactions on medical imaging 35(5), 1352–1363 (2016)

27. Nguyen, V., Pereira, L., Liang, Z., Mielke, F., Van Houtte, J., Sijbers, J., De Beenhouwer, J.: Automatic landmark detection and mapping for 2D/3D registration with BoneNet. Frontiers in Veterinary Science 9, 923449 (2022)

28. Nicolaou, A., Christlein, V., Riba, E., Shi, J., Vogeler, G., Seuret, M.: TorMentor: Deterministic dynamic-path, data augmentations with fractals (2022)

29. Paszke, A., Gross, S., Massa, F., Lerer, A., Bradbury, J., Chanan, G., Killeen, T., Lin, Z., Gimelshein, N., et al.: Pytorch: An imperative style, high-performance deep learning library. NeurIPS 32 (2019)
30. Pfandler, M., Stefan, P., Mehren, C., Lazarovici, M., Weigl, M.: Technical and nontechnical skills in surgery: A simulated operating room environment study. *Spine* **44**(23), 1396–1400 (2019)

31. Shrestha, P., Xie, C., Shishido, H., Yoshii, Y., Kitahara, I.: X-ray to CT rigid registration using scene coordinate regression. In: MICCAI. pp. 781–790 (2023)

32. Unberath, M., Gao, C., Hu, Y., Judish, A., Taylor, R., Armand, M., Grupp, R.: The impact of machine learning on 2D/3D registration for image-guided interventions: A systematic review and perspective. *Frontiers in Robotics and AI* **8** (2021)

33. Unberath, M., Zaech, J.N., Gao, C., Bier, B., Goldmann, F., Lee, S., Fotouhi, J., Taylor, R., Armand, M., Navab, N.: Enabling Machine Learning in X-ray-based Procedures via Realistic Simulation of Image Formation. *International journal of computer assisted radiology and surgery* **14**, 1517–1528 (2019)

34. Wang, J., Schaffert, R., Borsdorf, A., Heigl, B., Huang, X., Hornegger, J., Maier, A.: Dynamic 2-D/3-D rigid registration framework using point-to-plane correspondence model. *IEEE Transactions on Medical Imaging* **36**(9), 1939–1954 (2017)

35. Wolf, T., Debut, L., Sanh, V., Chaumond, J., Delangue, C., Moi, A., Drame, P., Adeline, J., Barrière, C., Rault, T., Louf, R., et al.: *HuggingFace's Transformers: State-of-the-art natural language processing*. arXiv:1910.03771 (2019)

36. Zaffino, P., Moccia, S., De Momi, E., Spadea, M.: A review on advances in intra-operative imaging for surgery and therapy: Imagining the operating room of the future. *Annals of Biomedical Engineering* **48**(8), 2171–2191 (2020)

37. Zhang, B., Faghihroohi, S., Azampour, M., Liu, S., Navab, N.: A patient-specific self-supervised model for automatic X-Ray/CT registration. In: MICCAI (2023)

38. Zheng, J., Miao, S., Wang, J., Liao, R.: Pairwise domain adaptation module for CNN-based 2-D/3-D registration. *Journal of Medical Imaging* **5**(2) (2018)