# Automated registration of intraoperative optical images with preoperative MRI

Charly Caredda, Eric Van-Reeth, Fernand Fort, Michaël Sdika, Thiébaud Picart, Jacques Guyotat, Fabien C Schneider, Bruno Montcel

► To cite this version:

Charly Caredda, Eric Van-Reeth, Fernand Fort, Michaël Sdika, Thiébaud Picart, et al. Automated registration of intraoperative optical images with preoperative MRI. European Conference on Biomedical Optics, Jun 2025, Munich, Germany. pp.M3A.15, 10.1364/ECBO.2025.M3A.15. hal-05385068

HAL Id: hal-05385068

https://hal.science/hal-05385068v1

Submitted on 27 Nov 2025

HAL is a multi-disciplinary open access
archive for the deposit and dissemination of sci-
entific research documents, whether they are pub-
lished or not. The documents may come from
teaching and research institutions in France or
abroad, or from public or private research centers.

L'archive ouverte pluridisciplinaire HAL, est
destinée au dépôt et à la diffusion de documents
scientifiques de niveau recherche, publiés ou non,
émanant des établissements d'enseignement et de
recherche français ou étrangers, des laboratoires
publics ou privés.

<figure><img src="image_1.png" /></figure>

Distributed under a Creative Commons Attribution 4.0 International License
Automated registration of intraoperative optical
images with preoperative MRI

Charly Caredda,¹,* Eric Van-Reeth,¹,² Fernand Fort,¹ Michaël Sdika,¹ Thiébaud Picart,³ Jacques Guyotat,¹,³ Fabien C. Schneider⁴ and Bruno Montcel,¹

¹ Univ Lyon, INSA-Lyon, Université Claude Bernard Lyon 1, UJM-Saint Etienne, CNRS, Inserm, CREATIS UMR 5220, U1294, F69100, Lyon, France

²CPE Lyon, Département Sciences du Numérique, Lyon, France

³Cranial Neurosurgical Department, Hospices Civils de Lyon, 69677 BRON, France - Lyon 1 University

⁴Université Jean Monnet Saint-Étienne, CHU de Saint-Étienne, TAPE Research Unit EA 7423, F-42023, Saint-Étienne, France

*charly.caredda@creatis.insa-lyon.fr

**Abstract:** This study presents a novel image processing pipeline for accurately registering pre- and postoperative brain MRI scans with intraoperative optical images of the cerebral cortex. This pipeline, seamlessly integrated into the clinical workflow, requires no additional devices during surgery. Compared to a landmark-based validation approach across nine patients, we achieved a mean registration error of 1.43 ± 0.72 mm. Furthermore, we observed a high degree of correspondence (Dice coefficient = 0.71) between functional activations identified with intraoperative optical imaging and preoperative fMRI data, demonstrating the clinical potential of this innovative approach. © 2025 The Author(s)

# 1. INTRODUCTION

Preoperative MRI, including T1-weighted and FLAIR sequences, is crucial for diagnosing brain tumors and guiding surgical planning. Functional MRI (fMRI) is used to map eloquent cortex, allowing radiologists and surgeons to assess the risk of neurological deficits and optimize the surgical strategy.

Craniotomy, a necessary step in neurosurgery, induces brain shift, significantly compromising the accuracy of preoperative neuronavigation for localizing functional areas [1]. While electrical brain stimulation (EBS) remains the gold standard for intraoperative mapping, its clinical application is limited by several factors, including low spatial resolution (≈ 5 mm [2]) and the inherent risk of triggering epileptic seizures.

Intraoperative optical imaging offers a promising approach to assess functional, metabolic, and morphological information of brain tissue during neurosurgery [3]. This technique allows for direct comparison with intraoperative electrical brain stimulation measurements, as the location of the imaged tissue is readily visible. To validate these tissue measurements, comparison with preoperative imaging modalities is crucial. However, accurate comparison with preoperative MRI acquisitions necessitates the development of robust co-registration techniques to account for brain shift and other surgical-induced deformations.

In the literature and in commercial systems, the comparison between optical measurements and preoperative MRI is conducted using a registration procedure based on landmarks [3, 4]. These “landmarks” denote identical locations in optical images and MRI volumes and are obtained with clinical neuronavigation systems. This registration approach has two main disadvantages: (1) it requires the intervention of the neurosurgeon with the use of a neuro-navigation tool and thus increases the duration of the surgery. (2) Locations indicated by neuro-navigation tools are not totally accurate. For some patients, we observed 2 cm shifts between points identified by the neuro-navigation tool and those validated by the neurosurgeon.

Recently, Villa et al. [5] proposed a novel technique to register hyperspectral images on MRI. The technique is based on the use of a multimodal system during neurosurgery. The system is composed of a RGB-Depth and hyperspectral cameras. The RGB-Depth camera captures the patient's facial geometry, which is used for registration with the preoperative MRI. Once MR depth registration is complete, the integration of HS data is achieved using a calibrated homography transformation. The hyperspectral camera is then moved to capture the craniotomy site. This technique aims to register optical images on MRI with a great precision (1.88 ± 0.19 mm) compared to the landmark-based technique (4.07 ± 1.28 mm). However, this technique has the major disadvantage of adding a
dedicated device in the operating room, which changes the clinical routine. Moreover, moving the optical system during the calibration phase can be very complex. Operating room space is limited and moving the optical system can compromise the sterile area. To overcome the limitations of current techniques, we propose to develop an automatic registration procedure that is fully integrated into clinical practice and does not require the addition of systems in the operating room.

## 2. MATERIAL AND METHODS

We represented the flowchart of the automatic registration pipeline in Fig. 1 A. This pipeline aims to register preoperative MRI acquisitions of the human brain with intraoperative optical images of the cerebral cortex. The registration process is based on the identification of key blood vessels, clearly visible on both contrast-enhanced T1-weighted MRI images and intraoperative optical images. The pipeline has been developed with Python with open-source frameworks.

<figure><img src="image_2.png" /><figcaption>Fig. 1. Registration of intraoperative optical image with preoperative MRI. A - Flowchart of data analysis, including definitions of the imaging modalities and image processing techniques used in the automatic registration pipeline. In the T1 projection, the blue contours represented the delineation of the tumor. The red points in the optical and T1 skeletons corresponded to points used to initialize the registration. B - Introoperative optical image registered on the T1 volume. The blue contour and the colormap represent the identification of the motor function obtained with preoperative fMRI and intraoperative optical imaging, respectively.</figcaption></figure>

### 2.1. Data acquisition

Preoperative FLAIR and fMRI volumes were registered on the T1 volume (rigid transform). Optical data were acquired during neurosurgery using RGB imaging with the wide field optical device described in [3] after the patient's craniotomy and before the brain tumor resection surgery. During the acquisition of optical and fMRI data, motor cortex stimulation was performed by the patient with repetitive, alternating hand opening and closing at ≈ 1 Hz. During the operation, the neurosurgeon also performed electrical brain stimulation to identify the eloquent cortex.

### 2.2. Registration pipeline

Using the T1 and the FLAIR volumes, a segmentation mask of the tumor was obtained with the BRATS toolkit [6]. A 2D image containing the projection of the cerebral cortex of the T1 volume was created. The orthogonal projection plane was defined as the plane whose normal passes through the center of gravity of the tumor and the origin of the MRI volume. This 2D projection of the T1 volume cerebral cortex is then compared with the 2D intraoperative optical image. The blood vessels were enhanced in the MRI projection and in the optical image with a Hessian-based Frangi vesselness filter [7]. Then, the blood vessels were segmented with a thresholding procedure and the topology of the blood vessels was extracted with a skeletonization algorithm. Using the skeletons of the blood vessels in the MR and optical images, the similarity transform that best match the two images was calculated with the Python itk framework [8] (metric: cross-correlation, optimizer: quasi-Newton method). The
registration procedure was initialized at 9 different locations located around the tumor to help the convergence.
The registration algorithm was characterized on 9 patients using a landmark-based validation approach. Finally,
functional identifications provided by fMRI and functional optics [3] were projected on the MRI volume and on
the optical image space. The comparison of the functional masks was calculated with a DICE coefficient.

3. RESULTS AND DISCUSSION

Results of the projection of the optical image on the MRI volume is presented on Fig. 1 B. With the landmark-
based validation approach, the average Euclidean distance between the optical and MRI validation landmarks is
1,43 ± 0,72 mm, which corresponds to an error of 2,77 ± 1,2 pixels. The mean registration error is less than the
mean error obtained by Villa et al (1.88 ± 0.19 mm) [5]. Although Villa et al obtained smaller standard deviation
error, our method has the advantage to be directly used in the clinical routine without using additional devices
during surgery. The DICE coefficient was computed between the binary functional masks obtained with fMRI
and optical imaging (see Fig. 1 B) and was 0.71, which indicates a good correspondence between the two binary
masks although using two different image acquisition methods. Using this registration pipeline, we plan to study in
future works the differences between fMRI and intraoperative functional imaging, especially in terms of functional
identifications and contrasts.

Acknowledgments

These works were funded by the European Union's Horizon Europe research and innovation programme under
grant agreement No 101071040 – project HyperProbe; LABEX PRIMES (ANR-11-LABX-0063) of Université
de Lyon, within the program “Investissements d'Avenir” (ANR-11-IDEX-0007), operated by the French National
Research Agency (ANR); Infrastructures d'Avenir en Biologie Santé (ANR-11-INBS-000), within the program
“Investissements d'Avenir” operated by the French National Research Agency (ANR) and France Life Imaging
(ANR-11-INBS-0006). This material is based upon work done on the ISO 9001:2015 PILoT facility.

**References**

1. Ian J. Gerard et al. Brain shift in neuronavigation of brain tumors: A review. *Medical Image Analysis*, 35:403–420, 1 2017.

2. J. Pallud et al. Technical principles of direct bipolar electrostimulation for cortical and subcortical mapping in awake craniotomy. *Neurochirurgie*, 63(3):158–163, 6 2017.

3. Charly Caredda et al. Intraoperative identification of functional brain areas with rgb imaging using statistical parametric mapping: Simulation and clinical studies. *NeuroImage*, 278:120286, 2023.

4. Martin Oelschlägel et al. Mapping of language and motor function during awake neurosurgery with intraoperative optical imaging. *Neurosurgical Focus*, 48(2):E3, 2020.

5. Manuel Villa et al. HyperMRI: hyperspectral and magnetic resonance fusion methodology for neurosurgery applications. *International Journal of Computer Assisted Radiology and Surgery*, 19:1367–1374, 2024.

6. Florian Kofler et al. Brats toolkit: translating brats brain tumor segmentation algorithms into clinical and scientific practice. *Frontiers in neuroscience*, 14:125, 2020.

7. Alejandro F Frangi et al. Multiscale vessel enhancement filtering. In *MICCAI'98, USA, October 11–13, 1998 Proceedings*, pages 130–137. Springer, 1998.

8. Denis P Shamonin et al. Fast parallel image registration on cpu and gpu for diagnostic classification of alzheimer's disease. *Frontiers in neuroinformatics*, 7:50, 2014.