# Optical Servoing of Soft Robotic Instrument for Cancer Imaging

Paul Chaillou, Jialei Shi, Alexandre Kruszewski, Isabelle Fournier, Helge Wurdemann, Thibaud Piccinali, Christian Duriez

► To cite this version:

Paul Chaillou, Jialei Shi, Alexandre Kruszewski, Isabelle Fournier, Helge Wurdemann, et al. Optical Servoing of Soft Robotic Instrument for Cancer Imaging. ICRA@40, Sep 2024, Rotterdam, Netherlands. hal-04714460

HAL Id: hal-04714460

https://hal.science/hal-04714460v1

Submitted on 30 Sep 2024

HAL is a multi-disciplinary open access
archive for the deposit and dissemination of sci-
entific research documents, whether they are pub-
lished or not. The documents may come from
teaching and research institutions in France or
abroad, or from public or private research centers.

L'archive ouverte pluridisciplinaire HAL, est
destinée au dépôt et à la diffusion de documents
scientifiques de niveau recherche, publiés ou non,
émanant des établissements d'enseignement et de
recherche français ou étrangers, des laboratoires
publics ou privés.

<figure><img src="image_1.png" /></figure>

Distributed under a Creative Commons Attribution 4.0 International License
# Optical Servoing of Soft Robotic Instrument for Cancer Imaging

P. Chaillou<fnref n="1" />,<fnref n="3" />, J. Shi<fnref n="2" />, A. Kruszewski<fnref n="1" />, I. Fournier<fnref n="3" />, H. A. Wurdemann<fnref n="4" />, T. Piccinali<fnref n="1" />, C. Duriez<fnref n="1" />

*Abstract*—This study presents a soft robotic instrument for in-vivo cancer detection during surgery, focusing on the precise servoing of a projected laser dot in space. Using optical servoing and closed-loop control, the system accurately positions the laser dot for tissue analysis, showcasing the robot's adaptability, even with external disturbances.

## INTRODUCTION

Cancer is an important cause of mortality. Being able to improve accuracy of surgical procedure to reduce post-operative effects is a major concern. Surgery is essential and remains the first frontline treatment of solid cancers. To determine the exact extension of the cancer and preserve non-cancerous tissue at most, robotic imaging can offer accurate scanning information to surgeons. In this work, we use robotic assisted mass spectrometry tissue surface analysis. Our goal is to develop an in-situ cancer detection during the surgery, so that surgeons can scan difficult-to-access area. The main challenge is to provide an easy and safe navigation in a tortuous anatomical environment

Our envisioned technology is based on the mini-invasive real-time mass spectrometry (SpiderMass Technology) [1], within the context of the Minimally Invasive Surgery (MIS). The SpiderMass Technology can analyse the cells in real-time and distinguish the different cell phenotypes, therefore, making it possible to discriminate types, subtypes and grades of solid tumors [2]. The contribution of this work is that we combine this real-time imaging technology with soft robots, aiming to achieve inherent-safe scanning. Our system might result in a significant reduction in operation time and impact on the patient's health. As proof of concept, optical servoing have been implemented, using a RealSense D405 3D camera. By estimating and correcting the projected laser dot position directly, we study the precision we could obtain for the laser shot used for the cells extraction before mass spectrometry analysis.

## MATERIALS AND METHODS

### A. System Overview

The soft robotic device is pneumatically-driven and each module features six circular reinforced chambers with a diameter of 1.5 mm. Two adjacent chambers are connected as a one pair. Two

<figure><img src="image_2.png" /><figcaption>Fig. 1 Illustration of the envisioned application: Developed soft robotic instrument for cancer imaging. A) Scheme that displays the soft robot in the MIS context. B) Picture that describes different configurations that the soft robot may achieve. C) Picture of the soft robot installed trough the trocar.</figcaption></figure>

modules are serially connected for a total length of 110 mm. The outermost diameter of the soft robot is 11.5 mm, designed to fit the commercially available 12 mm trocar port [3]. It is important to note that the robot's body in this work is constructed from silicone [Dragon Skin 10, Smooth-On, US] to enhance its stiffness and improve its force capability, while Ecoflex 10 [Smooth-On, US] is used for fabricating the chamber layer. During the operation, trocar seals prevent air from escaping the abdomen and maintain internal air pressure to provide surgeons with a clear operating area. However, friction of trocar seals leads to a buckling effect when inserting soft robot to the trocar port. As such, a sliding sleeve is designed to open the sealing gasket of the trocar, illustrated in Fig. 2.

### B. Modeling, Simulation and Control

The control of the robot is based on a simulation made using SOFA [4], as this software provides very useful tools for soft robotic modeling and control [5].

In order to reduce the computation cost of FEM simulation, the robot can be modeled through reduced FEM model [3]. Reducing computation cost increases the frequency of the close-loop correction, allowing a speed-up in the positioning control of the robot. The simulation is running at 11.64 FPS, instead of the 1.57 FPS of the Full-FEM simulation. This improvement demonstrates very interesting properties for biopsy retrieval, where it requires to reach point-per-point positions with sufficient accuracy.

<footnote n="1">¹Paul Chaillou, Alexandre Kruszewski and Christian Duriez are with the University of Lille, Inria, CNRS, Centrale Lille, UMR 9189 CRISTAL, F-59000 Lille, France.</footnote>

<footnote n="2">²Jialei Shi is with the Department of Mechanical Engineering, University College London. He is also with the Department of Mechanical Engineering, Imperial College London, UK.</footnote>

<footnote n="3">³Isabelle Fournier is with the University of Lille, Inserm, CHU Lille, U1192-Protéomique Réponse Inflammatoire Spectrométrie de Masses-PRISM, Lille F-59000, France.</footnote>

<footnote n="4">⁴Helge A. Wurdemann is with the Department of Mechanical Engineering, University College London, UK.</footnote>
<figure><img src="image_3.png" /><figcaption>Fig. 2 Picture of the experimental setup</figcaption></figure>

### C. Experimental Setup

The chamber pressure is regulated and monitored by solenoid valves [Rexroth ED02 R414002399]. These pressure regulators are controlled by proportional voltages, generated using a voltage output board [PhidgetAnalog 4-Output 1000_0b]. Pressurized air is supplied to the regulators by a compressor [Fengda FD-186]. An electromagnetic tracking system [Polhemus Liberty] is used as a moving target. A small red laser point 5mW is inserted trough soft robot central lumen. A depth camera [Intel Realsense D405] monitor the position of the laser dot in space.

### D. Experiment Methodology

This proof of concept seeks to validate the potential of use of soft robots for a new surgical outcome, but also more generally, to perform precise tasks. In this experiment it consists of positioning of a laser dot to a specified target. The robot's end-effector is separated from the target surface by some twenty centimetres. The position of the laser point is recovered by the depth camera through HSV filtering. Then, this position is used in a close loop control, to set the desired position of the soft robot end-effector. The actuation pressure is computed using inverse problem resolution available in the *SoftRobot.Inverse* SOFA plugin. The inverse problem is solved using an optimisation-based method [6], based on the reduced FEM model [3]. To avoid impossible desired position and simulation instabilities, the z dimension have been neglected, and only the x and y position have been controlled.

Multiple experiments have been conducted. Fisrtly with a static desired position, to demonstrate the capacity of the robot to target a specific location. Then, pertubations have been applied, to estimate the capacity of the control loop to correct unmodelled circonstancies. Finally, experiment with a magnetic target have been conducted, to estimate the capacity of the robot to aim a moving point.

## RESULTS

During static position targeting, once a steady state is achieved, the average positioning error is 2.64mm. This error can be attributed to the fact that a small displacement of the flexible robot results

<figure><img src="image_4.png" /><figcaption>Fig. 3 Curves that display the error between the laser measured and desired positions, for the static position experiment without perturbation.</figcaption></figure>

in a significant shift in the laser point position. Dynamic effects create vibrations in the system, which are one of the causes of residual error. Additionally, there is noise in the laser point positioning measurement, which is often overcompensated by the control loop. Modeling inaccuracies may also contribute to the error. The interplay of these various factors makes optimizing the control loop a complex task. The discrepancy between the desired and actual positions is illustrated in Fig. 3.

As shown in the videos, the system exhibits a strong ability to compensate for external forces applied to the soft robot, with the robot deforming more to counteract these forces.

The moving point experiment demonstrates the system's ability to adapt and remain effective across various robot configurations.

## CONCLUSION

This work presents the targeted surgical tissue scanning application and the soft instrument designed to retrieve biopsy samples. Then, we describe the setup and the simulation environment, which achieves inverse kinematics control depending on the desired trajectories. This setup is used in closed-loop experiment, to demonstrate the accuracy but also the limits of the system, as the simulation speed. This experiment is using a 11.64 FPS reduced-model simulation, and the performances could be optimized using a most powerfull hardware or a quicker modelisation.

SOFA is a fast way to compute FEM simulation [7], but also contains all the features to do real-time surgery simulation [8]. So, we may, in future works, simulate the robot and the surgical environment and the interaction between them.

## REFERENCES

[1] N. Ogrinc et al., "Robot-Assisted SpiderMass for In Vivo Real-Time Tomography Mass Spectrometry Imaging," *Analytical Chemistry*, vol. 93, 2021.
[2] P. Saudemont et al., "Real-Time Molecular Diagnosis of Tumors Using Water-Assisted Laser Desorption/Ionization Mass Spectrometry Technology," *Cancer Cell*, vol. 34, no. 5, pp. 840-851, 2018.
[3] P. Chaillou et al., "Reduced finite element modelling and closed-loop control of pneumatic-driven soft continuum robots," in *2022 IEEE 5th International Conference on Soft Robotics (RoboSoft)*, 2023, pp. 1-8.
[4] F. Faure, C. Duriez, H. Delingette, J. Allard, B. Gilles, S. Marchesseau, H. Talbot, H. Courtecuisse, G. Bousquet, I. Peterlik, and S. Cotin, "SOFA: A multi-model framework for interactive physical simulation," in *Soft Tissue Biomechanical Modeling for Computer Assisted Surgery*, Y. Payan, Ed.
[5] E. Coevoet, T. Morales-Bieze, F. Largilliere, Z. Zhang, M. Thieffry, M. Sanz-Lopez, B. Carrez, D. Marchal, O. Goury, J. Dequidt, and C. Duriez, "Software toolkit for modeling, simulation, and control of soft robots."
[6] C. Duriez, E. Coevoet, F. Largilliere, T. Morales-Bieze, and et al., "Framework for online simulation of soft robots with optimization-based inverse model," in *2016 IEEE International Conference on Simulation, Modeling, and Programming for Autonomous Robots (SIMPAR)*. IEEE, 2016, pp. 111-118.
[7] J. Allard, H. Courtecuisse, and F. Faure, "Chapter 21 - implicit FEM solver on GPU for interactive deformation simulation," in *GPU Computing Gems Jade Edition*, ser. Applications of GPU Computing Series, W.-m. W. Hwu, Ed.
[8] H. Courtecuisse, J. Allard, P. Kerfriden, S. P. A. Bordas, S. Cotin, and C. Duriez, "Real-time simulation of contact and cutting of heterogeneous soft-tissues."