# Robust laparoscope body calibration using hand-eye calibration methods

Chinmay Samant, Adlane Habed, Michel de Mathelin

► To cite this version:

Chinmay Samant, Adlane Habed, Michel de Mathelin. Robust laparoscope body calibration using hand-eye calibration methods. 9th Joint Workshop on New Technologies for Computer/Robot Assisted (CRAS 2019), Mar 2019, Gênes, Italy. pp.99-100. hal-04322464

HAL Id: hal-04322464

https://hal.science/hal-04322464v1

Submitted on 5 Dec 2023

HAL is a multi-disciplinary open access
archive for the deposit and dissemination of sci-
entific research documents, whether they are pub-
lished or not. The documents may come from
teaching and research institutions in France or
abroad, or from public or private research centers.

L'archive ouverte pluridisciplinaire HAL, est
destinée au dépôt et à la diffusion de documents
scientifiques de niveau recherche, publiés ou non,
émanant des établissements d'enseignement et de
recherche français ou étrangers, des laboratoires
publics ou privés.

<figure><img src="image_1.png" /></figure>

Distributed under a Creative Commons CC BY 4.0 - Attribution - International License
# Robust laparoscope body calibration using hand-eye calibration methods

Chinmay Samant¹, Adlane Habed¹, Michel de Mathelin¹

¹ICUBE, University of Strasbourg,
samant@unistra.fr

## INTRODUCTION

Ultrasound laparoscopes (US) are predominantly used in abdominal laparoscopic surgeries related to liver, pancreas and biliary tract. Despite being used in surgeries such as ablation procedures, they are harder to use for novice surgeons. The problem lies in understanding the proper orientation of the image with respect to the patient's body. Tracking the pose (position and orientation) of the laparoscope in real-time can address this problem. There are several solutions available [1,2,3] to this problem. All these solutions are based upon an Electromagnetic (EM) as well as optical pose tracker-based systems to track the pose of the US. In these methods, it is required to establish a chain of multiple transformations between the pose sensors. Some of them are fixed (do not change with the movement) and the rest are dynamic (change with a movement). The fixed transformations are between sensors which are mounted on the fixed part of the laparoscope as well as on the EM transmitters. The fixed transformations can be computed using 'Hand-Eye Calibration', as used in [1], which consists in estimating a fixed Euclidean transformation relating the reference frames of two rigidly attached sensors providing pose measurements. Since the EM sensors are constantly affected by different disturbances in the operation theatre, the measurements are often corrupted by noise and even worse, outlier data. Outliers are totally uncorrelated, non-synchronized measurements which together with the noise lead to faulty estimation of Hand-Eye Calibration. In this paper, we propose a robust method based on RANSAC [4] to get rid of the outlier data. We use a 'geometric' threshold for RANSAC based upon pixel error compared to the 'algebraic' one (in [8]). Due to this, we can comfortably find a value for the threshold for a wide range of datasets. We implement two state of the art methods in Hand-Eye calibration [5,6] in our RANSAC framework. In the results section, we compare the RANSAC implementations against the non-RANSAC versions of the methods.

## BACKGROUND

* Magneto-Optic Hybrid Tracking:

In a typical setup of the hybrid tracking (as shown in [1]), a very thin wired EM receiver is mounted on the tip of a laparoscope. It has its paired transmitter in the vicinity. Minimum two optical tracking elements are required in this setup. They both are tracked using a camera system with a view of the laparoscope. First optical tracking element is mounted on the handle of the laparoscope and the second is mounted on the EM transmitter. In [1], they use additional EM receivers on the body of the

laparoscope to estimate the transformation of the fixed body of the laparoscope. This is done by acquiring several poses of the EM sensors and optical trackers by moving the laparoscope in front of the camera and EM transmitter. The same poses are also used to compute the fixed calibration between the optical tracker on EM transmitter and the body of the EM transmitter itself. The poses are stacked into a system of equations which is used to solve the Hand-Eye Calibration.

* Hand-Eye Calibration:

The poses from the camera are stacked into $A$ and the poses from EM sensor are stacked into $B$. They both are related to each other by a system of equation $AX = XB$. $X$ is the fixed transformation. $A, B$ and $X$ are homogeneous transformations in $SE(3)$. Multiple recordings of $A$ and $B$ create an over-determined (with noisy data) system of equations. The data can also be infested with outliers especially if the data is recorded in an operation theatre with multiple sources of electromagnetic noises. We tested multiple methods of Hand-Eye calibration and selected [5,6] which are robust to the noise in the dataset. In [5], the authors use a locally optimal method to solve the problem and in [6], authors use a globally optimal method with quaternion parametrization of the rotation. We created a general RANSAC framework for Hand-Eye calibration in which these methods are incorporated.

## METHOD

Our RANSAC algorithm works in following steps.

1. Choose $n$ set of random samples from the dataset. In our case we select 4.

2. Compute Hand-Eye calibration using the methods [5,6]. Obtain the value of $X$.

3. Using $X$ and Hand motions, transform all the calibrated camera motions into each camera motion one-by-one. Compute the average pixel error between the obtained camera motion and the transformed camera motion. Based on this 'geometric' threshold, determine the consistent inlier pairs eliminating the outliers.

4. Retain the largest inlier set.

5. After all the inliers are found, re-compute the Hand-Eye calibration using the largest inlier dataset.

## EXPERIMENT AND RESULTS

We tested our algorithm extensively with synthetic as well as real dataset experiments. Due to the space constraints, we only present the results of the synthetic experiment with outlier infested dataset. The dataset is generated by creating random set of 3D points. 31 random camera motions are generated which capture the
3D points in each motion. A fixed calibration X is assumed (which is also the ground-truth) and 31 hand motions are created respectively. During the experiment, an increasing percentage of hand-motions are replaced with non-related wrong transformations to simulate outliers. We induce the outliers only in the hand-dataset since we believe that the EM sensors are mostly affected by them. We also introduce a nominal 0.5 Gaussian pixel noise on all the camera images, 0.01 absolute noise on quaternion transformations of hand dataset (as suggested in [7]) and 0.1% relative noise on the translation of the hand dataset. The probability value of the RANSAC is set to 0.99. Due to this, the RANSAC algorithm must find a solution if there are no time bounds, for any percentage of outliers in the dataset. But, in our experiments, we set a timer of 90 seconds after which the algorithm is forced-stop. Within this time our RANSAC algorithm finds a solution using at-least upto 50% of outlier infested dataset.

In the results, on the top, we show RMS of the errors in rotation unit quaternions $||q - q'||$ and in the middle, RMS of the relative errors in translation $||t - t'||/||t||$, where the $q'$ and $t'$ are the ground-truths. These are customary metrics used in [7] and other Hand-Eye calibration literature.

<figure><img src="image_2.png" /></figure>

## CONCLUSION AND DISCUSSION

In this paper, we presented a RANSAC framework for Hand-Eye calibration methods with a geometric threshold. Since the threshold is based upon pixel error compared to an algebraic error, it is easier to find a comfortable value for the threshold for a wide range of dataset. This framework can be used for any hand-eye calibration method with next to little modifications in the inputs to the methods. The presented results show that, the method can comfortably remove the outliers upto 50% within an acceptable computational time.

## REFERENCES

[1] Feuerstein, Marco, et al. "Magneto-optic tracking of a flexible laparoscopic ultrasound transducer for laparoscope augmentation." International Conference on Medical Image Computing and Computer-Assisted Intervention. Springer, Berlin, Heidelberg, 2007.

[2] Nakamoto, Masahiko, et al. "3D ultrasound system using a magneto-optic hybrid tracker for augmented reality visualization in laparoscopic liver surgery." International Conference on Medical Image Computing and Computer-Assisted Intervention. Springer, Berlin, Heidelberg, 2002.

[3] Konishi, Kozo, et al. "A real-time navigation system for laparoscopic surgery based on three-dimensional ultrasound using magneto-optic hybrid tracking configuration." International Journal of Computer Assisted Radiology and Surgery 2.1 (2007): 1-10.

[4] Fischler, Martin A., and Robert C. Bolles. "Random sample consensus: a paradigm for model fitting with applications to image analysis and automated cartography." Communications of the ACM 24.6 (1981): 381-395.

[5] Horaud, Radu, and Fadi Dornaika. "Hand-eye calibration." The international journal of robotics research 14.3 (1995): 195-210.

[6] Heller, Jan, Didier Henrion, and Tomas Pajdla. "Hand-eye and robot-world calibration by global polynomial optimization." Robotics and Automation (ICRA), 2014 IEEE International Conference on. IEEE, 2014.

[7] Daniilidis, Konstantinos. "Hand-eye calibration using dual quaternions." The International Journal of Robotics Research 18.3 (1999): 286-298.

[8] Furrer, Fadri, et al. "Evaluation of combined time-offset estimation and hand-eye calibration on robotic datasets." Field and Service Robotics. Springer, Cham, 2018.

<figure><img src="image_3.png" /></figure>

<figure><img src="image_4.png" /></figure>